Measurement Science for Manufacturing Robotics

Summary

Robotic systems continue to grow as essential tools in strengthening U.S. manufacturing competitiveness by enabling dramatically greater responsiveness and innovation. The criticality of robotics and related automation to the U.S. economy has increased in the past several years due to labor shortages, spurred by the COVID-19 pandemic and retirements in an aging U.S. workforce. Although certain core technologies underpinning advanced robotics capabilities have made strides in the past few years, adoption challenges persist, and new ones have emerged. Ensuring that robots are adaptable, easily tasked, can partner safely with humans, and can be quickly integrated into a manufacturing enterprise continues to be essential. Emerging technologies such as artificial intelligence, advanced sensors, and more dexterous manipulators hold great promise to expand and accelerate the adoption of robotics. However, this promise cannot be fulfilled without rigorous validations and characterizations of these technologies to ensure that they meet the applications and environments for which they are intended. The Measurement Science for Manufacturing Robotics program will provide the underpinnings needed to enable all manufacturers, including small and medium ones, to characterize and understand the performance of robotic systems within their enterprises. Measurement science establishes a common language for expressing performance requirements and provides means of verifying that systems meet those requirements. Tangible performance targets also direct innovations toward addressing existing capability gaps in robotic systems.  NIST will deliver performance metrics, information models, data sets, test methods, and protocols to assess and assure the key attributes of robotic systems necessary to enable enterprises of all sizes to achieve flexible and dynamic production.  

Program videos

NIST Robotics Wares: What You Can Use or Get Involved With Now

NIST NRC PostDoc Opportunity “Measurement Science for Manufacturing Robotic Systems”

Provide your Robotics Measurement Science and Standards Needs

Description

Objective
To develop and deploy measurement science that advances manufacturing robotic system performance, collaboration, agility, autonomy, safety, and ease of implementation to enhance U.S. innovation and industrial competitiveness.

Technical Idea
The fundamental idea is to provide the measurement science needed to ensure that robotic systems can be confidently applied to advanced manufacturing operations.  

As robots are systems of systems, to perform their assigned tasks, robots must perceive the world and their current state through a combination of sensors and algorithms. They must plan and adapt their actions based on their estimate of the current situation, and execute the plans using locomotion, grasping, and other actuation. They must also do this while interacting with humans and other robots and equipment through a variety of modalities. Artificial intelligence algorithms and data may support any of these systems. Robotic system performance is a composite of how well the components perform individually and as an integrated system.  Measurement science must be developed for the individual components as well as for the overall composite systems.  

Seven principal facets of robotic systems will be investigated through a holistic approach based on a unified set of testbeds and scenarios in consultation with industry. The capability-oriented research projects will be strengthened by a complementary effort identifying the technical barriers that prevent small and medium manufacturers from adopting robots, while serving as a conduit to the overall Program.  Underlying this approach is a process that uses application requirements to drive development of metrics.  Following this process provides performance results that are expressed contextually, rather than as abstract quantities or qualities that may not be relevant to intended implementations.  

The research is organized based on the core technical areas that underly manufacturing robotic systems. NIST will tackle new challenges within these technical areas, based on the highest priority needs identified by industry, to develop measurement science for characterizing and verifying the:

• Safe operations of robot arms and their tooling covered under new collaborative safety standards.  
• Effectiveness of new types of dexterous manipulators in handling a wider range of parts and tasks.
• Mobility and safety performance of automated industrial vehicles, mobile manipulators, and wearable robots within dynamic and unstructured environments.
• Ability of robots to perceive their surroundings in order to execute tasks correctly and safely despite variations.
• New modalities of human-robot and robot-robot interactions that enable safe and efficient teaming.
• Robot agility, meaning the ability to operate efficiently and effectively in a continuously-changing and unpredictable environment, and how to enhance it.
• Effective utilization of artificial intelligence and machine learning using validated and well-documented datasets and AI models.
• Intelligent in-situ health monitoring for decreasing the risks and costs of robot failures and degradation.
• Streamlined installation and integration of robots into workcells so that all U.S. manufacturers can realize the competitive advantages of robots.

Research Plan
The research plan’s thrusts outlined above are addressed in the following eight projects which share the Program’s testbeds and jointly work with industry to define relevant scenarios to drive the research. The areas below will advance understanding of the various component areas, based on technological and commercialization progress anticipated in the coming years.

1. Grasping, Manipulation, and Safety Performance of Robotic Systems 
   This thrust provides performance metrics, test methods, and associated measurement tools to support robot systems that have human-like dexterity and force control characteristics that enable tactile-based safe human-collaboration and manufacturing tasks.
2.   Perception Performance of Robotic Systems  
   The perception thrust develops measurement science for sensing and perception system performance characterization to reduce the risk related to the adoption of new technologies and to advance the agility, safety, and productivity of collaborative industrial and mobile robots in advanced manufacturing applications.
3. Mobility Performance of Robotic Systems 
   This thrust provides the measurement science to develop standard test methods for performance of intelligent industrial mobility systems, including mobile robots, mobile manipulators, and wearable robots (e.g., exoskeletons), to improve manufacturing flexibility and productivity.  
4. Performance of Human-Robot Interaction  
   This thrust provides test methods, protocols, and information models to facilitate effective human-robot collaboration in manufacturing and advance interactive robot technologies to enable manufacturers to leverage the safe and efficient teaming of people and robots toward meeting production goals.  
5. Agility Performance of Robotic Systems  
   This thrust delivers robot agility performance metrics, test methods, information models, data sets, and planning approaches that will enable manufacturers to more easily and rapidly reconfigure and re-task robot systems.
6. Embodied AI and Data Generation for Manufacturing Robotics  
   This project involves three main sub-thrusts: data generation, AI development, and downstream applications.
   The “data generation” effort focuses on identifying common data needs in the manufacturing robotics space and developing reproducible datasets and data collection procedures in response to these needs. The “AI development” effort focuses on training, deploying, and evaluating AI as part of a manufacturing robot system. The “downstream applications” effort develops downstream AI applications that target robot focus areas (such as Prognostic and Health Management) in collaboration with other Program focus areas, such as Agility, Perception, and Grasping.
7. Digital Twins and Emerging Technologies for SME Workcells
   This thrust determines the main obstacles to greater adoption of robotics by small manufacturers, provides measurement science that considers the unique perspectives of smaller enterprises, and generally acts as a conduit between the robotics program and small manufacturers.
8. Emergency Response Robots
   This project provides the measurement science necessary to quantitatively evaluate robotic system capabilities and to measure operator proficiency to advance the capabilities of emergency response robots.

Major Recent Accomplishments

Standards

• Exoskeleton: ASTM Committee F48
• Autonomous Unguided Ground Vehicle: ASTM Committee F45
• 3D Imaging: ASTM Committee E57
• Response Robots: ASTM Committee E54
• Robot Task Representation and Ontologies: IEEE RAS/SC
• Human-Robot Interaction: IEEE RAS/SC
• Industrial Robot Safety: ANSI/A3 R15.06 and ANSI/A3 R15.08
• NIST-developed reporting criteria adopted by the Association for Computing Machinery (ACM).

Roadmaps

• Contributed to multiples versions of the CCC Roadmap for U.S. Robotics (2013 – 2024)
• Standards Roadmap for 3D Imaging in Robotic Assembly Applications (2021)

Competitions

• RoboCup Federation RoboCup Rescue Competition (2001-present)
• NIST Robotic Grasping and Manipulation Competition (2016-present)
• NIST Unmanned Aerial Systems Flight and Payload Challenge (2018)
• NIST Agile Robotics for Industrial Automation Competition (ARIAC) - Annual Prize Competition (2018-present)
• IEEE ICRA Quadruped Robot Challenge (2023-present)
• ASTM International Exo Games (2023-present)

Testbeds

• Bin-Picking Testbed
• Robot Agility Testbed (physical and digital twin)
• Automated Vehicle Lidar Sensor Testbed
• Human-Robot Interaction Testbed
• Robot AI Learning Testbed
• Robot Performance and Health Management Testbed
• A-UGV Ramp Navigation Testbed

Patents

• Collaborative Robot Motion and Force-Torque Gauges (2018)
• Six-Dimensional Smart Target (2019)
• Low Cost System for Monitoring Degradation in Automation Systems (2019)

Datasets

• 3D Data for the Evaluation of Point-Based, Rigid Body Registration Error (2018)
• Data and Results from Three Peg-in-Hole Experiments for Improving Insertion Tasks (2018)
• Degradation Measurement of Robot Arm Position Accuracy (2019)
• Performance Data of a Robotic System with a Robotic Hand and a Robotic Gripper Completing a Peg-In-Hole Assembly Task (2019)
• Process and Robot Data From a Two-Robot Workcell Representative Performing Representative Manufacturing Operations (2021)
• Continuous Mobile Manipulator Performance Experiment (2022)
• Exoskeleton Performance Data (2022)
• Data for Filtering Organized 3D Point Clouds for Bin Picking Applications (2024)
• NIST Manufacturing Objects and Assemblies Dataset (MOAD) (ongoing)