EasyVizAR: Edge-Supported, Assistive Augmented Reality for Indoor First Responder Scenarios

University of Wisconsin System

In April 2021, University of Wisconsin (UW) was awarded $1.8M for the Public Safety Innovation Accelerator Program: Augmented Reality (AR) cooperative agreement. 

First responders work in highly challenging environments. They need to make split second decisions that may make the difference between life and death under stressful conditions. Every moment lost to issues in navigation, coordination between agencies, and situational awareness reduces triage response times and survivability rate for those involved in these dangerous situations. Easy, user-friendly access to information of different types can be the differentiator in many conditions. 

UW’s project focuses on new innovations in the use of AR systems, culminating in a system called EasyVizAR, that provides an end-to-end solution to address many challenges common in emergency situations, particularly  in indoor environments.

Meet the team

• Principal Investigator Suman Banerjee is a leader in the domain of mobile computing and wireless systems. He has led the development of some critical edge computing techniques to support efficient machine learning tasks for object detection, person identification, and more.
• Co-Principal Investigator Yuhang Zhao is an expert in human-computer interaction, augmented/ virtual reality, and assistive technology.

• Co-Principal Investigator Kevin Ponto has a rich history of working with 3D datasets and emerging technology and has led research efforts that utilize 3D scanning for purposes of crime scene investigation, cultural heritage, and the study of health in the home.

Project overview

Providing high-end computational support for AR devices is hard, due to processing costs that can either drain the battery fast or make the device particularly hot. UW’s project proposes the use of a portable on-premise edge-compute platform to offload computation. Further, it leverages recent advances in collaborative mapping and localization techniques in indoor environments to provide better navigational support for first responders. Finally, it utilizes novel multimodal feedback to assist users in accomplishing their tasks. Together, they provide a unique end-to-end system that is domain-optimized for challenging first responder scenarios.

Potential impacts of the project

The end-to-end EasyVizAR systems aims to help first responders:

• gain improved situational awareness and navigational abilities in indoor environments using conventional AR devices
• improve collaboration between multiple personnel, potentially from different agencies, in a secure and robust manner
• improve the battery life of AR devices to more optimally meet the above goals

The system also provides open application programming interfaces (API) by which information can be seamlessly integrated with various public safety databases. In doing so, EasyVizAR can improve key performance indicators of first responders, including triage response times and overall responsiveness to various emergency situations.

The team expects to build and release the entire EasyVizAR software infrastructure through the open source path to allow for flexible adoption and commercialization. The team also hopes to work with existing partners --- Office of Emergency Management (Milwaukee County), West Allis Fire Department, and Fox Valley Technical College’s Public Safety Technical Center --- to provide initial validation of developed technologies. Together this can provide an efficient pathway for adoption across a broader public safety community across the entire nation.