U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

An Analysis of Metrics and Methods in Research from Human-Robot Interaction Conferences, 2015-2021

Published

Author(s)

Megan Zimmerman, Shelly Bagchi, Jeremy Marvel, Vinh Nguyen, Xiang Li, Katherine Haas, Victoria Koretsky

Abstract

Standardized metrics and methods are critical towards wider adoption of HRI technologies in real-world applications. However, the interdisciplinary nature of HRI creates an inherently decentralized research paradigm that limits the use of standardized metrics for baseline comparisons among studies. This limitation restricts both the real-world adoption and academic replicability of HRI solutions developed by the research community. To identify specific opportunities for reuse of metrics and methods in HRI, this paper presents a comprehensive survey of 1464 papers from the ACM/IEEE International Conference on Human-Robot Interaction (HRI) and the IEEE International Conference on Robot and Human Interactive Communication (Ro-Man) over seven years. By providing a holistic perspective of the metrological tools leveraged in the current state-of-practice of HRI research, we find that a significant portion of HRI studies use custom surveys, thus limiting baseline comparison. Hence, the analysis in this work aims to advance the field of HRI by identifying specific barriers to adoption of HRI technologies in addition to proposing solutions to overcome existing limitations in the context of metrics and methodologies.
Proceedings Title
Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction
Conference Dates
March 6-12, 2022
Conference Location
Sapporo, JP
Conference Title
ACM/IEEE International Conference on Human-Robot Interaction
Pub Type
Conferences

Download Paper

https://doi.org/10.1109/HRI53351.2022.9889595
Local Download

Keywords

performance metrics, test methods, HRI metrology, research trends, reproducibility, replication, robotics
Robotics in manufacturing and Collaborative robots

Citation

Zimmerman, M. , Bagchi, S. , Marvel, J. , Nguyen, V. , Li, X. , Haas, K. and Koretsky, V. (2022), An Analysis of Metrics and Methods in Research from Human-Robot Interaction Conferences, 2015-2021, Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, Sapporo, JP, [online], https://doi.org/10.1109/HRI53351.2022.9889595, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934061 (Accessed November 20, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created March 6, 2022, Updated June 9, 2023