Agility Performance of Robotic Systems

Objective
Deliver robot agility performance metrics, test methods, information models, datasets, combined virtual/real testbeds, and planning approaches that will enable manufacturers to more easily and rapidly reconfigure, retask, and redeploy robotic systems.

Technical Idea
The project is developing the measurement science framework to enable manufacturers, integrators, and robot vendors to assess and assure the agility performance of their robotic systems. This framework will include:

1. Robot agility performance metrics and test methods,
2. Information models,
3. Datasets,
4. A combined virtual / real testbed (Digital Twin), and
5. Planning approaches.

Using the metrics, test methods, information models, datasets, and planning approaches developed by this project, manufacturers will be able to:

• Determine a robotic system’s ability to handle new assembly (and non-assembly) operations, and select the ‘correct’ robotic system for the job;
• Quantify a robot’s performance when confronted with unexpected events, obstacles, or failures;
• Quickly and easily re-task their robots when new assembly operations are needed (currently, reconfiguring a robot for a new task often takes an order of magnitude longer to program than the task would take to complete by hand);
• Assess a robot’s ability to cope with part and environmental variations (many of the current high-precision robotic systems are not able to adapt automatically to component-by-component or lot-by-lot variations); and
• Answer critical questions such as: “What does the robot need to know?”, “When does it need to know it?”, “How will it get that knowledge?”, and “How will the system adapt if that knowledge is different from expected?” (this knowledge is necessary for rapid re-tasking and will be enumerated and made explicit using the measurement science framework).

Research Plan
The project will focus heavily on manufacturing assembly applications, but also begin to explore other manufacturing domains that have significant importance to our customers, such as order fulfillment and logistics. The project will also expand the cross-collaboration with other projects in the Measurement Science for Manufacturing Robotics (MSMR) Program.

This project will focus on standards and performance metrics and will build upon our previous work in knowledge representation efforts in IEEE by continuing to contribute to more focused standards, such as the representation of robot tasks, a new implementation guide for using the Robot Task Representation standard, and a suite of test methods and metrics for measuring robotic agility. Standard planning languages, such as the Canonical Robot Command Language, will continue to be improved, by integrating and adapting it to work with ROS, and made available through the ROS-Industrial Consortium. All the standards developed in this project will be applied to industry-relevant scenarios and realized in the Robot Agility Lab and the Digital Twin of the Lab. These proof-of-concept implementations will help to validate the standards being developed and will demonstrate approaches by which industry can implement them.

Robot agility test methods and performance metrics will be exercised and will be validated through the Agile Robotics for Industrial Automation Competition (ARIAC). As they reach a suitable level of maturity, they will be incorporated into the ongoing IEEE Standards Association (SA) working group for Measuring Robot Agility.

The use of simulation paired with the physical Robot Agility Lab will be a large focus, initially by serving as a gateway to make the Robot Agility Lab available to the public as a testbed, allowing researchers to explore novel robot agility approaches and allowing the project to validate performance evaluation approaches. Through simulation software, ROS interfaces will allow external researchers to try out their robot agility approaches before implementing them remotely in the Robot Agility Lab.

Artificial Intelligence (AI) approaches will play a significant role in the project. Initial AI approaches applied to dynamic re-planning and reasoning over formal knowledge representations will be expanded into machine learning techniques applied to perception and path planning to design experiments that will yield AI training data. Focus areas will include, but not be limited to, learning from teach tables, optimizing action sequences for shorter run times, and making the robots easier to configure/program for novices.

A combined testbed for allowing multiple types of robots to be run in both the physical lab space, and a virtual twin of the lab space, combined with the emerging test methods and metrics, will allow researchers and industry to evaluate different approaches and will make the transition easier to industry use.

By the end of the five years, we plan to have Standards through the IEEE SA (including an implementation guide for the Robot Task Representation Ontology and an overarching Standard as well as a suite of more focused standards for Measuring Robot Agility), a Virtual/Real Testbed bridging the Robot Agility Lab and its Digital Twin to allow industry and researchers to test new approaches for agile solutions, and expand the ARIAC competition into assembly, logistics, and future ideas based on collaborations with industry and within the MSMR Program.

Major Accomplishments

• Agile Robotics for Industrial Automation Competition (ARIAC) - Annual Prize Competition since 2018
• IEEE Standards Association Robotics and Automation Society Standards and Working Groups
  • 1872-2015 - Standard Ontologies for Robotics and Automation
  • 1872.1-2024 - Standard for Robot Task Representation
  • P1872.1.1 - Working Group for Implementation Guide for Robot Task Representation Standard (anticipated 2026)
  • P2940 - Working Group for Measuring Robot Agility (anticipated 2024-2025)

Publications

• Robotics and Computer Aided Manufacturing Journal - Special Issue on Knowledge Driven Robotics and Manufacturing (June 2015) 
• Test Methods for Robot Agility in Manufacturing (June 2016) - Industrial Robot Journal
• Industrial Robot Journal Special Issue on Industrial Robot Agility (September 2016)
• Towards a Robot Task Ontology Standard (MSEC 2017 - June 2017)
• Using Standards in a Competition to Measure and Solve Industrial Robotics Agility Challenges (MSEC 2017 - June 2017)
• Overview of an Ontology based Approach for Kit Building Applications (November 2017)
• Implementation of an Ontology based Approach to Enable Agility in Kit Building Applications (December 2018)
• A Suite of Ontologies for Robotics and Automation (March 2017)
• Enabling Robot Agility in Manufacturing Kitting Applications (March 2018)
• The Agile Robotics for Industrial Automation Competition (December 2018)
• Special Issue on Enabling Robot Autonomy (March 2018)
• Agile Industrial Robots (January 2022)
• Physics-Based Simulation of Agile Robotic Systems (November 2019)
• Recent Advances in Industrial Robotics (March 2021)
• Assessing Industrial Robot Agility Through International Competitions (August 2021)
• Runtime Verification of the ARIAC Competition: Can a Robot be Agile and Safe at the Same Time? (November 2020)
• Agile Robotics for Industrial Applications Editorial (March 2021)
• Agility Metrics in the ARIAC Competition (July 2021)
• Agile Robotic Planning with Gwendolen (May 2022)
• Using a BDI Agent to Represent a Human on the Factory Floor of the ARIAC 2023 Industrial Automation Competition (September 2023)