NIST Performance Tests for Aerial Response Robots Become National Standard

Tracking the spread of a wildfire. Searching for a missing person. Detecting toxic fumes after an explosion before sending in rescue workers.

These are just three of the many tasks being assigned to flying robots known as small unmanned aircraft systems (sUAS) or emergency response drones. In such critical situations, the performance of the robot and the pilot remotely controlling it must be flawless. To help ensure that capability, a new standard was published recently based on test methods developed by the National Institute of Standards and Technology (NIST).

Sky's the Limit: NIST Performance Tests Train Emergency Response Drone Pilots

The National Fire Protection Association (NFPA), a U.S. organization that creates and maintains more than 300 codes and standards to reduce loss of life and property due to fire, electrical and related hazards, has issued NFPA 2400, Standard for Small Unmanned Aircraft Systems (sUAS) Used for Public Safety Operations. The new standard includes a suite of 10 aerial test methods developed by NIST that quantitatively measures both the system capabilities of the drone and the proficiency of the pilot in carrying out five basic maneuvers, including accurate landing, vertical climbing, and straight and level flying. There also are five functionality test methods, including circular orbits to identify objects from afar and spiral maneuvers to conduct close-range inspections. 

“NFPA 2400 references our test methods as a practical way to guide training and measure remote pilot proficiency,” said Adam Jacoff, NIST robotics research engineer. Jacoff leads the agency’s efforts to develop, refine and promote adoption and use of consensus test standards for response robots that function on land, sea and air.  

“The test methods are low-cost and easy to fabricate and use, so they can be replicated by any group interested in evaluating its aircraft and remote pilots,” he said. 

“Emergency response organizations can quantitatively establish reliability and confidence during training before drones and remote pilots are deployed in real-life situations,” said NIST mechanical engineer Kamel Saidi, who works with Jacoff on metrics for response robots. “For the aircraft, it’s like testing a car on a speedway track to make sure it works as designed. For the pilot, it’s like a driving test to prove he or she can operate a specialized vehicle that’s not for everyone, such as a commercial truck.” 

Many of the test methods involve aligning the drone simultaneously with two cylindrical targets in a series of designated positions and orientations. “Primarily, these tests are implemented using white buckets with black rings inscribed inside them,” Jacoff said. “The pilot uses onboard cameras to align with and identify the targets, perform a task such as reading a hazardous material label or detecting motion, and then move on to the next drill.”

 

 

Another test has the remote pilot prove his or her skills at avoiding obstacles by performing figure eight transits through a course of hurdle-like gates with vertical posts and horizontal crossbars.

More than 20 additional aerial test methods are being developed by NIST. These are all being standardized through the ASTM International Standards Committee on Homeland Security Applications: Response Robots (E54.09), which is chaired by Jacoff. Included are tests for navigating hallways within buildings, searching and mapping wide areas, and avoiding situations that may interfere with radio communications.

The new standard and its planned expansions are being welcomed by first responders as valuable resources.

“The tests give you specific numbers that quantify the whole process: the aircraft, the pilot, everything,” said Victor Galladora, a Montgomery County, Maryland, police officer who has helped validate the NIST test method suite operating emergency response drones at NIST’s Robotics Test Facility. “These metrics allow you to say with confidence that a drone works well, or someone is a really good pilot. And when a test becomes standardized, it means you can be in Los Angeles and I can be here in Maryland and we can both get the same training, optimize our performances and then compare results.”

NFPA 2400 is available on the organization’s Codes and Standards website. More information on NIST’s standard test methods for emergency response robots, as well as the Robotics Test Facility, may be found online.