Evaluating automatic face recognition systems with human benchmarks

Published

April 9, 2015

Author(s)

P. Jonathon Phillips, Alice O'Toole

Abstract

Human face recognition skills are often considered the gold standard against which machines must compete. Over the last two decades, however, international tests of computer-based face recognition algorithms have shown steady improvements in accuracy with increasingly challenging photometric conditions. Indeed, the most recent comparisons between humans and algorithms show that the best algorithms compete favorably with humans recognizing frontal images of faces—even across substantial changes in illumination, facial expression, and appearance. We review these comparisons considering both quantitative and qualitative benchmarks for evaluating performance on identification tasks. We also address the question of how to statistically fuse the judgments of humans and machines to improve performance over either "system" operating alone. On the qualitative dimension, studies have shown that the long-standing human challenge of recognizing people of different races and ethnicities has parallels in machine vision. We discuss complex problems this poses for predicting how well computer-based systems will operate in environments with variable demographic diversity (e.g., airport). In summary, we argue that computer-based face recognition systems are now at the level of humans recognizing unfamiliar faces. The next challenge for machines is to begin to operate with the accuracy and robustness humans show for familiar face recognition.

Citation

Forensic Facial Identification: Theory and Practice of Identification from Eyewitnesses, Composites and CCTV

Publisher Info

John Wiley & Sons, Ltd, West Sussex, -1

Pub Type

Book Chapters

Download Paper

DOI Link

Citation

Phillips, P. and O'Toole, A. (2015), Evaluating automatic face recognition systems with human benchmarks, Forensic Facial Identification: Theory and Practice of Identification from Eyewitnesses, Composites and CCTV, John Wiley & Sons, Ltd, West Sussex, -1, [online], https://doi.org/10.1002/9781118469538.ch11 (Accessed October 31, 2024)