Face Challenges

Introduction

In cooperation with IARPA, NIST is currently running three challenges related to processing of unconstrained in-the-wild face images.  These are intended to drive research and development into face detection, verification, identification, and identity clustering.  A secondary goal is to establish definitive evaluation metrics and reporting.

NIST has discontinued distribution of all three IJB challenges (IJB-A, IJB-B, and IJB-C) on March 14th 2023.

IJB-C

The IARPA Janus Benchmark-C face challenge (IJB-C) defines eight challenges addressing verification, identification, detection, clustering, and processing of full motion videos.  This is supported by the IJB-C set of 138000 face images, 11000 face videos, and 10000 non-face images.  Challenge  Documentation

Face Recognition Prize Challenge

From June to September 2017, NIST evaluated 41 face recognition algorithms from 16 developers. The algorithms were applied to datasets of 2D still photographs in two ways: verification of “wild” photojournalism and social media images, and identification of faces from surveillance videos against portrait galleries of size up to 691 thousand. The outcomes of the Face Recognition Prize Challenge (FRPC) is documented in NIST Interagency Report 8197.

IJB-B

The IARPA Janus Benchmark-B face challenge (IJB-B) defines eight challenges addressing verification, identification, detection, clustering, and processing of crowded images.  This is supported by the IJB-B set of 67000 face images, 7000 face videos, and 10000 non-face images.  

IJB-B has been superseded by IJB-C.  Dataset    Challenge Documentation 

IJB-A

The IARPA Janus Benchmark-A face challenge (IJB-A) was an open challenge in which researchers executed algorithms on NIST-provided image sets, and returned output data to NIST for scoring.  From 2015-2017 NIST produced a results report. That process was ineffective at tracking state-of-the-art as most groups using the IJB-A dataset did not submit to NIST. The report is still available on-request. Readers interested in robust comparison of face recognition algorithms should see results from the FRVT evaluations: 1:1 and 1:N.

IJB-A has been superseded by IJB-B. Its images form a subset of IJB-B.   Dataset     CVPR paper

FRVT

The Face Recognition Vendor Test is an ongoing evaluation of face recognition algorithms applied to large image databases sequestered at NIST.  Algorithms may be submitted to NIST at any time, and results will be posted when ready, usually within two weeks.   Homepage