NIST logo
Bookmark and Share

Robotic Systems for Smart Manufacturing Program


To develop and deploy advances in measurement science that enhance U. S. innovation and industrial competitiveness by improving robotic system performance, collaboration, agility, and ease of integration into the enterprise to achieve dynamic production for assembly-centric manufacturing.


Due to their inherent flexibility and reusability, robotic systems are an essential tool in strengthening U. S. manufacturing competitiveness by enabling dramatically greater responsiveness and innovation. To attain these gains, robotic systems need to be highly-capable, perceptive, dexterous, and mobile systems that can operate safely in collaboration with humans, are easily tasked, and can be quickly integrated into the rest of the enterprise. The Robotic Systems for Smart Manufacturing program will provide the measurement science needed to enable all manufacturers, including small and medium ones, to characterize and understand the performance of robotics systems within their enterprises. Measurement science establishes a common language for expressing performance requirements and provides means of verifying that systems meet those requirements. Concrete performance targets also direct innovations towards addressing existing capability gaps in robotic systems. NIST will deliver performance metrics, information models, test methods and protocols to assess and assure the key attributes of robotic systems necessary to enable this new dynamic production vision.

What is the problem?

Agile and collaborative robotic systems are a disruptive technology essential to achieving a new vision of manufacturing because of their tremendous flexibility coupled with high-precision and repeatability. Yet, it is estimated that only 10% of potential users in the manufacturing domain have adopted robotic systems. This is because there is a lack of measurement science infrastructure to assure manufacturers that robotic components and systems can be readily integrated into their operations and will perform as needed under dynamic unstructured shop floor conditions.

"Improved productivity in the increasingly competitive international environment" is one of 3 key factors driving the adoption of robots cited in a the Computing Community Consortium (CCC) Roadmap for U.S. Robotics – a key document guiding the White House's Office of Science and Technology Policy views on robotics R&D. The President's Framework for Revitalizing American Manufacturing recognizes the importance of "developing advanced robotics technologies that allow the U.S. to retain manufacturing and respond rapidly to new products and changes in consumer demand". The PCAST report on Ensuring American Leadership on Advanced Manufacturing called out robotics as an "important technology area for advanced manufacturing." The CCC Robotics Roadmap notes that "Robotics is a key transformative technology that can revolutionize manufacturing… the promise of flexible automation and automation for mass customization has not been realized except for special cases…. Robots [need] to be smarter, more flexible, and able to operate safely in less structured environments shared with human workers." A 2013 workshop on Opportunities in Robotics, Automation, and Computer Science sponsored by CCC, NSF, OSTP, and the Robotics-VO summarized some of the barriers to greater adoption of advanced robotics by industry. The report noted the lengthy and expensive process of installing robots as well as designing and implementing assembly lines due to lack of equipment and control models and tools. Another key challenge is the inability to transfer successful components and solutions to other manufacturing applications or across equipment due to lack of component modularity and integration and interoperability standards. These barriers can be overcome with NIST-led development of measurement processes for creating and refining models of robotic system equipment and assembly operations and knowledge representation and component interface standards.

Progress towards having robots fulfill their potential within manufacturing facilities has been hindered the lack of metrics, benchmarks, test methods, reference architectures, and standards. Research institutions achieve advancements in sensor fusion, situational awareness, autonomous planning, grasping, navigation, and other capabilities essential to attaining a future vision of robotic systems for smart manufacturing. Yet few of these advancements make their way into commercially-available industrial robots. The CCC Robotics Roadmap highlights the need for a measurement science infrastructure to help transition research into products and reduce the risk of adopting new robotic technologies. This lack of measurement science infrastructure results in barriers and inefficiencies in (a) expressing end user performance requirements for new robotic capabilities; (b) assessing progress in robotic capabilities towards meeting industry needs; (c) validating new technologies for deployment by manufacturers, and (d) enabling interoperability and ease of integration of robotic systems and components.

What is the new technical idea?

The fundamental idea is to focus on the measurement science needed to ensure that robotic systems can be confidently applied to smart manufacturing assembly-centric operations. Four principal facets of robotic capabilities will be investigated, while taking a holistic approach in having a unified set of testbeds and assembly-centric scenarios developed jointly with industry. These four capability-oriented research projects will be strengthened by a complementary venture focusing on reducing the technical barriers small and medium enterprises encounter today installing and using robots.

First, the overall robotic system performance must be assessed and assured. This entails being able to characterize the performance of the foundational constituent capabilities of the robotic system – perception, mobility, dexterity, and safety – and being able to compose these into an overall system performance model that provides manufacturers with currently-missing data to reduce the risk of adopting this key disruptive technology.

Robots must function as trusted co-workers, alongside humans, as well as being able to collaborate with other robots to accomplish tasks. This aspect is lacking test methods, protocols, and information models to assess and assure the collaborative performance whilst achieving assembly performance objectives.

Wider use of robotics in manufacturing, especially within assembly, is hindered by their lack of agility, their lengthy changeover times for new tasks and new products, and their limited reusability. This program will provide manufacturers with an integrated agility assessment framework so that they can evaluate how well a robotic system will be able to function within their application environment.

The fourth aspect focuses on integration and interoperation and will address the obstacles to easily integrating robotic assembly systems within manufacturing facilities. Models of the underlying information required to automate the composition and integration of complex robotic assembly systems, along with a suite of tools to foster interoperability will address the existing incompatibility between robots and the next generation of perception, mobility, and manipulation technologies needed to achieve automated assembly.

Complementing the above efforts is a one that tackles the technically challenging procedures that hinder adoption of robotic systems by small and medium enterprises. Specifically, the calibrations of robot arms, sensors, and end-of-arm tooling are essential procedures when installing new systems and must be executed periodically thereafter to maintain correct performance. A tool suite that automates the creation of the complex models and parameters necessary to achieve correct robotic system performance will enable easier installation and greater robustness during the life of the robotic system.

What is the research plan?

The research plan has five research thrusts, which will share the Program's testbeds and jointly work with industry to define relevant scenarios to drive the research.

  1. Performance Assessment Framework for Robotic Systems. This thrust will enable manufacturers to assess and assure robot system assembly task performance by delivering a methodology and tools for characterizing and composing performance of perception, mobility, dexterity, and safety components. In first year, we will produce a robot systems capability model detailing a taxonomy of assembly tasks, decomposed into sub-tasks to define performance metrics and measurement methods for the abovementioned components of the robot system. This capability model will guide and focus the future development of robot systems for use in assembly-centric manufacturing. Test method suites of increasing complexity will be developed in the subsequent four years, along with a methodology and tools for composing their results into a systems-wide model.
  2. Performance of Collaborative Robot Systems. This thrust will deliver a suite of test methods, protocols, and information models to assess and assure that robots working collaboratively, with and without humans-in-the-loop, will complete their assigned tasks correctly while meeting their assembly performance objectives. Four task areas comprise this research plan: evaluating the coordinated performance of robot systems, decomposing collaborative tasks to model human and robot role representations and assignments, developing protocols for robot-robot and human-robot collaboration communications, and validating the situational awareness of collaborative robot systems.
  3. Agility Performance of Robotic Systems. This thrust will deliver robot agility performance metrics, information models, test methods and protocols, validated using a combined virtual and real testing environment, that will enable manufacturers to easily and rapidly reconfigure and re-task robot systems in assembly operations. This thrust will develop the deliverables through a research plan with four tasks: test methods and metrics, formalized robot description and environmental models of manufacturing assembly operations, a prototype planning infrastructure including information models and planning protocols, and the validation of the outputs of the prior three tasks via a real and virtual environment.
  4. Robotic Systems Interoperability and Integration. This thrust will enable manufacturers to quickly and easily put together robotic assembly systems with human-like capabilities using interoperable components by delivering an analysis, information model, protocol, reference implementation, test methods, and tools. Starting with an assessment of the state of the art in technology for robotic assembly, this thrust will develop new information models and communications protocols and build reference software implementations to validate the models and protocols within the Program's testbeds.

Major Accomplishments:

Some recent accomplishments for the Agility Performance of Robotic Systems:

  • CRCL specification released and widely available on ROS-I distribution.
  • An IEEE Industrial Ontology Study Group approved by IEEE and convened to advance the development of the industrial ontology.
  • Core Ontologies for Robotics and Automation becomes an IEEE Standard (1872-2015).
  • Special Issue of Robotics and Computer Integrated Manufacturing Journal (Guest Editor: Craig Schlenoff) published containing four refereed paper from Agility Team members.

Some recent accomplishments for the Performance Assessment Framework for Robotic Systems:

  • was created; test methods were demonstrated at ICRA 2015; paper documenting metrics/tests accepted by IEEE RAM December 2015. 
  • Methodology for assessing human detection and tracking perception systems.
  • Industry and academia adopt NIST-led test methods and standards to evaluate candidate perception technologies and guide engineering decisions.
  • Standard methodology for evaluating pose determination systems and comparing alternative systems.