NIST Proprietary Fingerprint Template (PFT) Evaluation 2003 -2010

The NIST PFT (2003 - February 2010) evaluation is no longer accepting SDKs for evaluation it has been replaced with PFTII evaluation.

Since June of 2003, NIST has been conducting evaluations of fingerprint-based biometric matching systems using vendor supplied SDKs and comparing these systems to our internal fingerprint matching system. Fingerprint matching systems from vendors are being evaluated to insure that the accuracy of the matchers used in various existing and planned government systems (including US-VISIT) are comparable to the most accurate available COTS products. The main result obtained from these evaluations is an estimate of how well commercial products performed one-to-one matching for verification over a wide range of fingerprint image qualities. The relative accuracy of thumbs and index fingers is also investigated.

Unlike the Fingerprint Vendor Technology Evaluation, FpVTE , these evaluations are ongoing and new SDKs can be included in the test at any time. There are two additional differences between the PFT tests and FpVTE. First, the PFT evaluations are performed on larger samples of each dataset. Second, during the FpVTE testing, each vendor was provided a set of N images and asked to return a matrix of NxN matched scores. The program to perform the testing in FpVTE was written by each vendor and run on the vendor's own computer hardware. In PFT testing, each call to the matcher function compares two fingers at a time and returns a score to the application that was written by NIST. The application that performs the matches uses SDK libraries supplied by each vendor and run on NIST computer hardware.

The final report on the first round of the PFT evaluations performed at NIST is available. This report describes the testing procedures used to evaluate each vendor's matching algorithms. Performance ROC plots are included for each SDK tested. The same techniques for testing and presenting results as described in this report will be used to evaluate any algorithms that are submitted to NIST for testing. The report name is NISTIR 7221 "Studies of One-to-One Fingerprint Matching with Vendor SDK Matchers".

In order to assist you in the testing of your algorithm before submitting it to NIST, we can also provide you with a CD-ROM of fingerprint images. This CD contains fingerprint images from about 180 different subjects. For each subject a pair of tenprint records are included. Each record has 10 rolled images, two plain thumb images, and eight plain images segmented from the two 4-finger slap images. This amounts to having two flat and two rolled images from each finger. With the use of the fingerprint images on this CD you can determine the performance of your system and compare it to those ROC plots of the same data from other systems reviewed in the NISTIR 7221 report.

This can be a very valuable tool for testing your algorithm. It can build confidence in your product. On the other hand, if your algorithm is not ready for NIST evaluation, it can save you the time and resources involved with our testing procedure. 

Results:

• NISTIR 7249 (July 2005) - Two Finger Matching with Vendor SDK Matchers.

Updated Tables and Plots (April 5, 2010).

• Tables
• Dataset Plots
• SDK Plots
• NISTIR 7221 (April 2005) [62K] - Studies of One-to-One Fingerprint Matching with Vendor SDK Matchers,
• Appendix A [53K] - Datasets ranked for each SDK,
• Appendix B [240K] - SDKs ranked for each dataset,
• Appendix C [5,213K] - ROC plots for each SDK,
• Appendix D [3,795K] - ROC plots for each dataset,
• Appendix E [94K] - ROC datapoints for each SDK,
• Appendix F [67K] - ROC datapoints for each dataset.

Current List of Vendor Names Associated with SDK Letters

These tests were performed for the Department of Homeland Security in accordance with section 303 of the Border Security Act, codified at 8 U.S.C. 1732. Specific hardware and software products identified in this report were used in order to adequately support the development of technology to conduct the performance evaluations described in this document. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products and equipment identified are necessarily the best available for the purpose.

PFT links

Home
PFTII
FAQ
Testing Status
API Specifications