R. Madhavan, Oak Ridge National Laboratory, Oak Ridge, TN, USA;
E. Tunstel, Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA;
E. Messina, National Institute of Standards and Technology, Gaithersburg, MD, USA
Performance Evaluation and Benchmarking of Intelligent Systems presents research dedicated to the subject of performance evaluation and benchmarking of intelligent systems by drawing from the experiences and insights of leading experts gained both through theoretical development and practical implementation of intelligent systems in a variety of diverse application domains. Scientific methodologies for standardization and benchmarking are crucial for quantitatively evaluating the performance of emerging robotic and intelligent systems' technologies. This contributed volume offers a detailed and coherent picture of state-of-the-art, recent developments, and further research areas in intelligent systems.
The chapters cover a broad range of applications, such as assistive robotics, planetary surveying, urban search and rescue, and line tracking for automotive assembly. Subsystems or components described in this book include human-robot interaction, multi-robot coordination, communications, perception, and mapping. Chapters are also devoted to simulation support and open source software for cognitive platforms, providing examples of the type of enabling underlying technologies that can help intelligent systems to propagate and increase in capabilities.
Performance Evaluation and Benchmarking of Intelligent Systems serves as a professional reference for researchers and practitioners in the field. This book is also applicable to advanced courses for graduate level students and robotics professionals in a wide range of engineering and related disciplines including computer science, automotive, healthcare, manufacturing, and service robotics.
Researchers and practitioners in the field of intelligent systems and robotics
Chapter 1. Metrics for Multiagent Systems
Chapter 2. Evaluation Criteria for Human-Automation Performance Metrics
Chapter 3. Performance Evaluation Methods for Assistive Robotic Technology
Chapter 4. Issues in Applying Bio-Inspiration, Cognitive Critical Mass and Developmental-Inspired Principles to Advanced Intelligent Systems
Chapter 5. Evaluating Situation Awareness of Autonomous Systems
Chapter 6. From Simulation to Real Robots with Predictable Results: Methods and Examples
Chapter 7. Cognitive Systems Platforms using Open Source
Chapter 8. Assessing Coordination Demand in Cooperating Robots
Chapter 9. Measurements to Support Performance Evaluation of Wireless Communications in Tunnels for Urban Search and Rescue Robots
Chapter 10. Quantitative Assessment of Robot-Generated Maps
Chapter 11. Mobile Robotic Surveying Performance for Planetary Surface Site Characterization
Chapter 12. Performance Evaluation and Metrics for Perception in Intelligent Manufacturing
Chapter 13. Quantification of Line Tracking Solutions for Automotive Applications