Response Robot Evaluation Exercise (#7)

Project Overview

The National Institute of Standards and Technology (NIST), with sponsorship from the Department of Homeland Security Science and Technology Directorate, is developing a suite of DHS-NIST-ASTM International Standard Test Methods for Response Robots to quantitatively evaluate the performance of robots for emergency response applications. These standard test methods measure robot capabilities in mobility/maneuvering, energy/power, sensing, radio communication, manipulation, human-robot interaction, logistics, and safety to provide point of comparison for a variety of robot sizes and configurations prior to testing in more realistic scenarios. Statistically significant robot performance data captured within standard test methods measure incremental system improvements, highlight break-through capabilities, and support procurement and deployment decisions. More than thirty such test methods are under development with associated apparatuses, procedures, and performance metrics. They are being standardized through the ASTM International standards committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01).

This suite of standard test methods is being developed to measure the range of capabilities of response robots independent of robot size. The working definition of a "response robot" is a remotely deployed device intended to perform operational tasks at operational tempos that can serve as an extension of the operator to improve remote situational awareness, provide means to project operator intent through the equipped capabilities, improve effectiveness and efficiency of the mission, and reduce risk to the operator. Key features include:

• Rapidly deployed
• Remotely operated from an appropriate standoff
• Mobile in complex environments
• Sufficiently hardened against harsh environments
• Reliable and field serviceable
• Durable and/or cost effectively disposable
• Equipped with operational safeguards

These test methods address high-priority tasks identified by emergency responders including: fast, light and mobile reconnaissance tasks for throwable robots; wide area survey tasks for hazmat or other events for packable or luggable robots; counter IED, VBIED, and PBIED tasks for mobile manipulators; aerial reconnaissance for small unmanned aerial systems (sUAS) conforming to the emerging FAA Group I class weighing less and 2kg (4.4lbs), less than 30 knots maximum speed in horizontal flight, and frangible or otherwise harmless upon impact; underwater reconnaissance for small unmanned underwater vehicles (sUUV) or remotely operated vehicles (ROV). For each of these application domains the standard test methods enable quantitative robot evaluations, provide practice tasks, and measure operator proficiency.

The standards development process involves hosting periodic robot requirements workshops, standards committee meetings, and response robot evaluation exercises at responder training facilities to gather emergency responders, robot developers, and test administrators around draft standard test methods and practice deployment scenarios. These events allow emergency responders to articulate essential robot capabilities, validate proposed test methods, and refine performance thresholds and objectives based on objective performance data captured across a class of robots. Emergency responders involved in the process learn about the state-of-the-science in robotic capabilities and help ensure that the test method apparatuses and procedures address their application needs. These events also inform robot developers regarding the reliability and applicability of their robots for actual deployment scenarios, and the ease of use of their systems as they train responders within the test apparatuses. Robot developers involved in the process learn about emerging operational requirements and can demonstrate robotic capabilities by capturing statistically significant performance data within the resulting standard test methods.

The seventh in a series of DHS/NIST Response Robot Evaluation Exercises will be hosted at the emergency responder training facility known as Disaster City in College Station, TX. Thirty emergency responders from across the country will participate -- half representing FEMA urban search and rescue teams and half representing bomb squads from across the country. They will help validate emerging standard robot test methods, become familiar with available robot capabilities, and advise robot developers regarding operational requirements. All applicable robots are invited to take part in these exercises including ground, aquatic, and small VTOL aerials under 2kg (4.4lbs). Robots will capture performance data within emerging standard test methods, familiarize and train responders within the test methods, then be deployed with responders to perform operational tasks in practice scenarios including:

• Test methods for mobility, obstacles, endurance, radio communications (line of sight, non line of sight, and structure penetration), sensors (video acuity, pan-tilt-zoom tasks, 2-way speech intelligibility, range imager resolution, thermal imager resolution) will prepare robots to perform operational tasks for down-range reconnaissance of a hazardous materials and passenger train wrecks from stand-offs greater than 150m/500ft.
• Test methods for navigating, searching, and mapping (2D and 3D) complex environments will prepare robots for operational tasks in building interiors and exteriors, partially collapsed structures, and confined spaces in rubble piles.
• Test methods for mobile manipulation (non-contact inspection, access tool for window breaking and boring, and grasping/removal tasks) will prepare robots for operational tasks to counter improvised explosive devices (C-IED), vehicle-born improvised explosive devices (C-VBIED), and personnel-borne improvised explosive devices (C-PBIED).
• Test methods for towing trailers and gripper-dragging objects will prepare robots for operational tasks in C-IED, C-VBIED, C-PBIED and US&R scenarios.
• Test methods for underwater navigation, station-keeping, and sensor acuity will prepare for operational tasks in vehicle reconnaissance in the on-sight pond.
• Test methods for air-worthiness, station-keeping, and sensor acuity will prepare small unmanned aerial systems with vertical take-off, hovering, and landing capabilities for operational tasks supporting several scenarios noted above.
  

These response robot evaluation exercises introduce emerging robotic capabilities to emergency responders within training facilities, while educating robot developers regarding the performance requirements necessary to be effective. They also help correlate the draft standard test methods with envisioned deployment tasks and lay the foundation for usage guides identifying a robot's applicability to particular response scenarios. The result will be validated or refined draft standard test methods, quantitative robot performance data captured to support balloting within the ASTM International Committee on Homeland Security, Operational Equipment (E54.08.01), and an updated Response Robot Capabilities Compendium to inform responders and soldiers regarding what robots can and cannot be expected to do reliably in the field.

These events help refine the proposed standard test apparatuses, procedures, and metrics while allowing robot developers to practice critical capabilities and measure performance in ways that are relevant to emergency responders. The associated responder training scenarios help correlate the proposed standard test methods with envisioned deployment tasks and lay the foundation for usage guides identifying a robot's applicability to particular response scenarios.

These events help refine the proposed standard test apparatuses, procedures, and metrics while allowing robot developers to practice critical capabilities and measure performance in ways that are relevant to emergency responders. The associated responder training scenarios help correlate the proposed standard test methods with envisioned deployment tasks and lay the foundation for usage guides identifying a robot's applicability to particular response scenarios.

Disaster City is a 52-acre training facility designed to deliver the full array of skills and techniques needed by urban search and rescue professionals. As part of the Texas Engineering Extension Service (TEEX) at Texas A&M University and a training site for TX-TF1, the facility features full-size collapsible structures that replicate community infrastructure, including a strip mall, office building, industrial complex, assembly hall/theater, single family dwelling, train derailment, three rubble piles, a C-VBIED scenario, and underwater vehicle reconnaissance scenario.