Robot Perception for Identifying and Locating Parts for Assembly

Perception systems underpin many advanced, autonomous, industrial-robot capabilities.  These advanced capabilities include manipulation of unfixtured parts. This project will develop measurement science and standards to support advances in perception that can improve ways of locating and identifying parts for assembly and sensor calibration and registration. 

Objective

To develop test methods for evaluating the performance of perception systems used for locating and identifying unfixtured, manufacturing parts under static conditions by the end of FY 2014.

What is the new technical idea?

Discrete manufacturing processes such as assembly, bin picking, and kitting require the ability to determine the positions and orientations (poses) of parts in unstructured manufacturing environments in order for robots to manipulate them.  Improvements to the perception systems that measure part poses are needed  to make them more robust, accurate, flexible and efficient.  A flexible perception system can adapt to new conditions through the use of calibration and registration techniques.  Similarly, an efficient system can provide pose estimates of parts in real time.  However, essential technologies, including new sensors and algorithms, are maturing slowly and the standards and test procedures needed to speed maturation do not exist.

The new technical idea is to develop the needed measurement science by focusing on the development and validation of test methods, including uncertainty calculations, to measure industry-chosen performance metrics for sensors.  These research activities will be conducted jointly with end users, technology providers, and academia.  The results will be demonstrated in the testbed established as part of the Robot Perception for Workspace Situational Awareness Project and disseminated in the form of standards through ASTM.

What is the research plan?

The research plan concentrates on developing methods for evaluating a perception system’s ability to identify static parts and measure their locations and orientations.  The allowed uncertainties in the location and orientation measurements are application-dependent and defined by the users. In this project, the parts can be isolated in space, in bins, or positioned in both structured and unstructured factory floor environments.

Developing experimental and computational methods for assessing ground truth system uncertainty is a foundational activity for this project.  These systems are critical to the generation and validation of new test methods and part datasets or artifacts needed to develop performance metrics and support standards development in perception systems.  Since these systems rely on sensor calibration and sensor-to-robot and sensor-to-sensor registration, it is important to understand how to quantify the uncertainties associated with these calibration and registration algorithms.

Developing test methods to evaluate the performance of part identification systems requires an understanding of the kinds of manufacturing parts that are typically used in assembly applications.  A classification of manufacturing parts into groups of parts with common characteristics is a critical first step.  This will allow for the development of test methods for specific classes of parts.

In developing these new experimental and computational test methods, the project will rely on research results from FY12.  It will use a proposed performance evaluation standard for static, six degree of freedom (6DOF) localization. This standard is being developed through ASTM committee E57, with NIST leadership.  In FY13, performance metrics and test methods to support standards for static pose estimation systems under varying environmental conditions will be developed.  Also in FY13, a draft test method will be developed to evaluate the volumetric performance of 3D imaging systems which are used for part identification and localization.  In FY14, the project will evaluate pose estimation systems in more realistic conditions such as parts in bins and will develop metrics and methods for evaluating the performance of part identification systems.

Second, the project will use the ground truth systems developed as part of the testbed to develop metrics and test methods for performance of perception systems for pose estimation in FY13.

Third, the project will continue efforts begun in FY12 to develop methods to calibrate and register ground truth systems and develop a method for determining the uncertainty of a part’s pose measured with a ground truth system. These new techniques are needed to support perception system evaluations, which will be done in FY14.

Recent Results:

• Output – 2 journal papers, 4 conference papers, and one NIST report :
  
• • Franaszek, M., “Registration of Six Degrees of Freedom Data with Proper Handling of Positional and Rotational Noise,” submitted to IEEE Transactions and Pattern Analysis and Machine Intelligence (TPAMI).
  • Franaszek, M., “Variances of Cylinder Parameters Fitted to Range Data”, submitted to JRES NIST
  • Shah, M., Eastman, R., and Hong, T., “An Overview of Robot-Sensor Calibration Methods for Evaluation of Perception Systems”, in PerMIS 2012 proceedings.
  • Marvel, J., Falco, J., and Hong, T., “Ground Truth for Evaluating Six Degrees of Freedom Pose Estimation Systems”, in PerMIS 2012 proceeding.
  • Marvel, J., Eastman, R., Cheok, G., Saidi, K., Hong, T., Messina, E., Bollinger, B., Evans, P., Guthrie, J., Hershberger, E., McNamara, K., and Wells, J., “Technology Readiness Levels for Randomized Bin Picking Report, NIST-TR, 2012.
  • Marvel, J., Eastman, R., Cheok, G., Saidi, K., Hong, T., and Messina, E., “Technology Readiness Levels for Randomized Bin Picking,” in PerMIS 2012 proceeding.
  • Marvel, J, Franaszek, M., Wilson, J., Hong, T. and Chang, T., “Performance Evaluation of Consumer Grade 3D Sensors”, SPIE Optical Engineering Applications conference, Aug. 2012.
  • MacKinnon, D., Beraldin, J-A, Hong, T., and Marvel, J., “Coordinate Metrology and the Proposed E57.02 Static Pose Determination Standard", Coordinate Metrology Systems Conference (CMSC) CMSC, 2012

Standards and Codes

• Outcome:  Draft standard for performance evaluation of static pose estimation systems will be submitted for ballot through ASTM E57 in September 2012.
• Outcome:  Draft standard for performance evaluation of the relative range of 3D imaging systems will be submitted for ballot through ASTM E57 in September 2012.