Face Recognition Vendor Test (FRVT) 2013

FRVT - Publications

FRVT Gender Classification Report
April 20, 2015: The first public report on gender classification has been published as NISTIR 8052- Face Recognition Vendor Test (FRVT) Performance of Automated Gender Classification Algorithms.

FRVT One-to-Many Face Recognition Report
May 21, 2014: We are pleased to announce that the report of the FRVT's class C track evaluating one-to-many face recognition algorithms has been released. The document, NIST Interagency Report 8009, gives core technical performance of face recognition algorithms used to identify up to 1.6 million individuals. It updates a similar report from our 2010 evaluation.

FRVT Age Estimation Report
March 20, 2014: The first public report on age estimation has been published as NISTIR 7995 - Performance of Automated Age Estimation Algorithms.

FRVT - Ongoing Updates

July 25, 2013: FRVT is proceeding in five classes of participation. Participation is now open for Phase 3 of the classes A (one-to-one), C (one-to-many), and D (pose, sex, age estimation). Algorithms may be submitted from July 25 until the hard deadline of October 4, 2013. For the new class F frontal reconstruction task the schedule for the first phase of evaluation is July 25 until September 12, 2013. The class V recognition from video activity is ongoing with the phase 2 schedule yet to be announced.

The API specification is here.

Jan. 8, 2013: The FRVT evaluation has attracted unprecedented interest and is ongoing. The revised schedule is as follows. For Classes A, C and D (which process still images): All Phase 1 submissions should be received by January 24. The last Phase 1 report will be sent to participants by February 4. Phase 2 will begin on February 5, and all algorithms should be sent to NIST by March 28. The Phase 3 schedule will be announced at the end of Phase 2. New and current participants are welcome to submit algorithms to Phases 1, 2, or 3.

For Class V (video sequences): Phase 1 is open now, and will remain open until at least until Feb 8. The Phase 2 schedule will be announced at end of Phase 1, and a Phase 3 will follow.

Nov. 28, 2012: Updates to the FRVT 2013 Video validation package are now available at the following link https://nigos.nist.gov/frvt2012/video/frvt2012VideoValidation_updates.tar. The tar file contains two updated source files (frvt2013Video.h and .cpp), which should be copied into the validation package's src/include and src/testdriver folders respectively.

These updates have also been rolled into the complete FRVT 2013 Video validation package, which is still available here https://nigos.nist.gov/frvt2012/video/frvt2012VideoValidation.tar.gz.

Oct 12, 2012: The FRVT 2013 Video validation package is now available https://nigos.nist.gov/frvt2012/video/frvt2012VideoValidation.tar.gz . The purpose of validation is to ensure that NIST's execution of your SDK submission produces the expected output. The process requires you to

1. compile and link the provided validation code against your libraries,
2. run the executables, and
3. email the results to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov).

Please see the included README file for more details. NIST will run the same validation code on your submitted SDKs and check that the results come out identical. Comments regarding the source code should be directed to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov).

Aug 15, 2012. A updated FRVT 2013 Concept and API document clarifies on the number of class C that may be submitted. Two kinds of class C implementation are defined – class CN and CP which are intended to minimize different costs applied to false negative and false positive errors respectively.

July 31, 2012. Additional information has been provided in an updated version of the FRVT 2013 Concept and API document.

July 10, 2012: The FRVT 2013 validation package is now available. The purpose of validation is to ensure that NIST's execution of your SDK submission produces the expected output. The process requires you to 1) compile and link the provided validation code against your libraries, 2) run the executables, and 3) email the results to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov).

Please see the included README file for more details. NIST will run the same validation code on your submitted SDKs and check that the results come out identical. Comments regarding the source code should be directed to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov) .

July 2, 2012: The final version of the Concept, API and Evaluation plan is now available. It includes revised participation details in sections 1.11 and Annex A2. In addition, the final version of the formal participation agreement is also available.

June 15, 2012: NIST announces the availability of the third draft Concept, Evaluation Plan, and API for FRVT 2013. The document is linked here NIST_FRVT2013_api_v07.pdf. Comments and questions should be directed to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov).

May 15, 2012: NIST announces the availability of the second draft Concept, Evaluation Plan, and API for FRVT 2013. The document is linked here NIST_FRVT2013_api_06.pdf. Comments and questions should be directed to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov).

April 17, 2012: NIST announces the availability of the concept, evaluation plan and API for the conduct of the FRVT 2012 evaluation. The activity is intended to evaluate face recognition algorithms with still and video imagery. The API document NIST_FRVT2013_api.pdf is now available. The first draft of this document is open for public comment until April 30, 2012.

Prospective participants in the FRVT2012 evaluation should read, sign, and submit the participation agreement linked from FRVT2013_ParticipationApplication_v1.pdf. Questions should be directed to frvt2013 [at] nist.gov (frvt2013[at]nist[dot]gov)

Introduction

The Face Recognition Vendor Test (FRVT) 2013 continues a series of large scale independent evaluations for face recognition systems. Previous activities in the series were the FERET, FRVT 2000, FRVT 2002, FRVT 2006 and MBE 2010. The primary goal of the FRVT 2013 is to measure advancements in the capabilities of prototype systems and algorithms from the commercial and academic communities. The FRVT 2013 aims to evaluate performance on several recognition tasks, including:

One-to-many identification in a set of mugshot images
One-to-one verification of visa images
Multi-sample still facial imagery
Recognition of persons in video sequences
Twins
90 degree profile-view
Gender estimation (M|F)
Age estimation (in years)
Pose estimation
Expression neutrality

Schedule

The FRVT 2013 will commence in late July 2012. Algorithms can be submitted to NIST from that time until mid December, at which time NIST will prepare test reports for publication starting by mid 2013.

Procedures and Interface Specifications

The test will be conducted according to a participation agreement, procedures and an API specification that will be released in late March 2012. This will be derived from that used for the 2D still image track of the MBE 2010.