First Responder Intelligent Assistant (FRIA)

TRACLabs, Inc.

In June 2022, TRACLabs was awarded nearly $1.2M for the Public Safety Innovation Accelerator Program 2022 (PSIAP-2022) funding opportunity. 

Sensor technology provides an abundance of data from cameras and other hardware and software sources in emergency scenarios. However, too much data is produced too fast for the systems to manage. To become actionable information, this data needs to be sorted, filtered, put in context and presented to the right user at the right time, in the optimal format for action. FRIA, First Responder Intelligent Assistant, reduces information overload and increases operation tempo by improving data management and access using context-aware systems.

Meet the team

TRACLabs’ team is made of the following key personnel:

• Debra Schreckenghost, Principal Investigator, Senior Scientist at TRACLabs, a recognized expert in human interaction with intelligent systems and robots.  She has led projects for NASA, NSBRI, and DoD, most recently serving as PI of the virtual intelligent task assistant (VITA) for NASA to assist crew in performing procedural work.
• David Kortenkamp, Co-Principal Investigator, Senior Scientist at TRACLabs, a recognized expert in intelligent systems. He has led projects for NASA, DoD, and NIST, most recently serving as Principal Investigator of the PSCR project Virtual and Augmented Laboratory for Objective Realities (VALOR).
• John Blackwell, Co-Principal Investigator, Founder of Diamond Age Technology developing spatial-computed solutions, including virtual and augmented reality and immersive interfaces. He has led projects for the NSF, DoD, and NIST.  He was recently awarded a NASA STTR to develop digital-twin simulations for human exploration of the Moon and Mars.       
• Patrick Hagen, Subject Matter Expert Consultant, Emergency Operations Technical Specialist Captain of the Houston Fire Department (HFD).
• Edward Wright, Subject Matter Expert Consultant, Officer in Philadelphia Police Department (PPD), a 12-year veteran of the Philadelphia Police Department who is currently assigned to the Federal Bureau of Investigation (FBI) violent gang task force. 

Project overview

TRACLabs is partnering with Porter Fire Department to develop a First Responder Intelligent Assistant (FRIA), a technological solution that improves access and dissemination of information for decision makers, asset operators, and front line responders. FRIA will synthesize, structure, and route actionable insights from the sUAV data and then present it in context to first responders and machine systems in an interaction format that fits their needs. 

TRACLabs’ solution provides three complementary but independently operable systems:

1. A spatio-temporal system that provides location based indexing and networking of information so that long streams of video and other content can be leveraged quickly and to maximum effect by accessing the short video clip relevant to the specific area of interest. In the long term, this technology also allows us to build a real-time, evolving digital-twin of the environment; 
2. Location-based push notifications that provide an automated system for determining which information is relevant to whom, when, and where. This is not intended to override the system of hierarchical command. It is a tool to disseminate information about the evolving scene without overloading the end-user, and by possibly providing that lifesaving critical piece of information front and center when it is needed most; and 
3. A context-aware multimodal system which provides an automated system for managing the multiple information input/output options that place stress on a position such as a technical specialist, who may be interfacing with multiple human and machine systems. In some sense, it provides a universal remote for the many disparate technologies a first responder may have to learn to interface with and remember to switch between during high-pressure situations.

Potential impacts of the project

FRIA will impact first response operations through a series of advancing implementations, beginning with solutions that are deployable using current hardware already in the hands of first responders, and advancing with the implementation of augmented and virtual reality interfaces, wearable technology, and robotic assets. This technology is able to scale the network bandwidth requirements as well to reduce total information exchange to the critical components and to take advantage of 5G networks to transmit more robust data when available. FRIA will provide a managed, spatially, and contextually aware information system capable of collecting, collating, and disseminating information across human and machine teams. This tool will improve the speed of planning, execution, and event response as well as enable improved audits, after action analysis, and reporting with a greater capacity to share experiences and lessons learned.