Take a sneak peek at the new NIST.gov and let us know what you think!
(Please note: some content may not be complete on the beta site.).

View the beta site
NIST logo
Bookmark and Share

2003 Performance Metrics for Intelligent Systems (PerMIS'03) Workshop


In the fourth workshop in a series targeted at defining measures and methodologies of evaluating performance of intelligent systems, we will examine more closely applications of performance measures to practical problems in commercial, industrial, and military applications. In the quest for providing researchers, users, and developers of intelligent systems with meaningful and usable measures and methodologies, we will attempt to draw upon measurement technologies and practices from other disciplines.

Papers and invited sessions are being sought. Topic areas include, but are not limited to

Leveraging Measures from other disciplines: 

  • Technology Readiness Levels (TRL)
  • Uncertainty Measures
  • Complexity Measures, such as Kolmogorov
  • Biometric Approaches
  • Linguistic Approaches
  • Levels of Autonomy
  • Cognitive Science Approaches
 Measuring Components of Intelligent Systems
  • Sensing and Perception
  • Modeling and Knowledge Content, Representation
  • Planning and Control
  • Learning and Adapting
  • Communications with Humans (and Other Systems)
  • Collaboration with Other Systems (and Humans)

Code of Best Practice for Experimentation
Testbeds and Competitions for Inter-comparisons
Tools for Facilitating Performance Measures
Evaluating Architectures for Intelligence


Past Workshops


Start Date: Tuesday, September 16, 2003
End Date: Thursday, September 18, 2003
Format: Workshop


NIST, DARPA, IEEE Control Systems Society, IEEE Systems, Man, and Cybernetics Society in cooperation with IEEE Neural Network Council.

Call for Papers (pdf)

Plenary Addresses

PerMIS 2003 Agenda (pdf)

Technical Contact:

Elena Messina, NIST
Alex Meystel, Drexel University
Dennis K. Leedom, Evidence Based Research Inc.

Advisory Board

  • G. Adorni, University of Parma, Italy
  • J. Albus, NIST, USA
  • P. Antsaklis, University of Notre Dame, USA
  • M. Asada, Osaka University, Japan
  • G. A. Bekey, University of Southern California, USA
  • K. Bellman, Aerospace Integration Science Corp., USA
  • J. G. Blitch, SAIC, USA
  • P. Borne, Ecole Centrale de Lille, France
  • H.-H. Bothe, Technical University of Denmark, Denmark
  • B. Chandrasekaran, Ohio State University, USA
  • J. Cherniavsky, NSF, USA
  • M. Cotsaftis, LTME/ECE, France
  • R. Cottam, ETRO VUB, Belgium
  • F. Darema, NSF, USA
  • P. Dario, Scuola Superiore, Italy
  • P. Davis, RAND Graduate School., USA
  • J. Fetzer, University of Minnesota, USA
  • D. Filev, Ford, USA
  • R. Finkelstein, Robotic Technology, Inc., USA
  • D. Fogel, Natural Selection, Inc., USA
  • N. Foo, University of New South Wales, Australia
  • W. Freeman, University of California at Berkeley, USA
  • E. Fromm, Drexel University, USA
  • T. Fukuda, University of Nagoya, Japan
  • D. Gage, DARPA, USA
  • R. Garner, Loebner Prize Winner for 1998 and 1999, USA
  • G. Gerhart, US Army TACOM, USA
  • E. Grant, CRIM, North Carolina State University, USA
  • S. Grossberg, Boston University, USA
  • R. Gudwin, State Univerity of Campinas, Brazil
  • W. Hamel, University of Tennessee, USA
  • W. Hargrove, Oak Ridge National Laboratory, USA
  • M. Herman, NIST, USA
  • E. Horvitz, Microsoft Research, USA
  • M. Jabri, University of Sydney, Australia
  • D. Jaron, Drexel University, USA
  • A. Jones, NIST, USA
  • R. Jordan, Lockheed Martin, USA
  • C. Joslyn, Los Alamos National Laboratory, USA
  • S. Kak, Louisiana State University, USA
  • O. Kaynak, Bogazici University, Istanbul, Turkey
  • H. Kitano, Sony Computer Science Labs, Japan
  • K. Kreutz-Delgado, University of California at San Diego
  • F. Kurfess, California Polytechnic State University
  • J. E. Laird, University of Michigan, USA
  • C. Landauer, Aerospace Integration Science Corp., USA
  • S. Lee, Samsung Advanced Inst. of Technology, Korea
  • C. S. George Lee, Purdue University, USA
  • P. B. Luh, University of Connecticut, USA
  • B. Mirkin, Birkbeck College, GB
  • U. Ozguner, Ohio State University, USA
  • T. Parisini, Politecnico di Milano, Italy
  • K. Passino, Ohio State University, USA
  • L. Perlovsky, AFRL/SNHE, USA
  • L. Pouchard, Oak Ridge National Lab, USA
  • J. Pustejovsky, Brandeis University, USA
  • D. Repperger, AFRL/HECP, USA
  • E. H. Ruspini, SRI International, USA
  • T. Samad, Honeywell, USA
  • A. Sanderson, RPI, USA
  • R. Sanz, University of Madrid, Spain
  • G. Saridis, RPI, USA
  • A. Schultz, Naval Research Laboratory, USA
  • T. Shih, Tamkang University, Taiwan
  • R. Simmons, Carnegie Mellon, USA
  • M. Swinson, Sandia National Lab, USA
  • H. Szu, ONR Navy, USA
  • M. Tilden, Los Alamos National Lab., USA
  • S. Tzafestas, National Techical University of Athens, Greece
  • L. Tsoukalas, Purdue Universtity, USA
  • I. B. Turksen, University of Toronto, Canada
  • C. Weisbin, NASA, USA
  • T. Whalen, Georgia State University, USA
  • A. Wild, Motorola, USA
  • V. Winter, University of Omaha, USA
  • R. Yager, Iona College, USA
  • A. Yavnai, RAFAEL, Israel
  • Y. Ye, IBM T. J. Watson Research Center, USA
  • B. Zeigler, University of Arizona, USA
  • L. Zadeh, University of California at Berkeley, USA