IREX 10: Performance Metrics

OVERVIEW

This page provides a high-level explanation of common performance metrics presented in IREX 10 reports. IREX 10 is a technology evaluation that does not assess performance for a specific deployment. The metrics are generic so as to convey a broad sense of algorithmic capabilities. It should be noted that biometric decision making is often analogous to the problem of binary classification, and as such no single numerical value can comprehensively describe the accuracy of a biometric system.

Identification Systems

Due to its scalability, iris recognition is well-suited for large deployments ranging from border security, prison management, expedited processing, duplicate detection, and distribution of resources. Most of these applications utilize iris recognition in  identification mode. Biometric identification systems search users against a database of enrolled identifies and return their proper identities when found.

Accuracy for the identification task cannot be fully characterized by a single numerical value. Two distinct types of error are possible. The first, a false negative, is roughly analogous to a "miss". In an access control scenario, a false negative would deny access to an authorized user. The other type of error, a false positive, occurs when a searched individual is falsely associated with one of the enrolled identities. This would grant access to an unauthorized user for the access control scenario. The decision criteria of the system can be adjusted to reduce the rate at which one type of error occurs, but at the expense of increasing the other. In the leaderboard, algorithms are compared by fixing the rate of false positives (FPIR) and reporting the rate of false negatives (FNIR). The full relationship between the two error rates is characterized by a DET plot.

Some publications report a matcher's rank one "hit rate". This performance metric is relevant to applications where a human reviewer will always review the candidates produced in an automated iris search. This is typical in law-enforcement investigations.

Equal Error Rates are deprecated in IREX 10 because they report accuracy at operating thresholds that are typically far from what would be used in practice.

Contact Information

Inquiries and comments may be submitted to irex [at] nist.gov (irex[at]nist[dot]gov).  Subscribe to the IREX mailing list to receive emails when announcements or updates are made.