NIST logo

Human Assisted Speaker Recognition Evaluation

Summary:

The goal of the NIST Human Assisted Speaker Recognition (HASR) Evaluation series is to contribute to the direction of research efforts that begin to address the question: "How can human experts effectively utilize automatic speaker recognition technology?", and to begin addressing issues that are important for forensic applications.

The HASR task is: Given two different speech segments determine whether they are both spoken by the same speaker, utilizing any approach and any combination of human experts and technology.

Description:

The 2010 SRE evaluation (SRE10) included a test of Human Assisted Speaker Recognition (HASR), in which systems based, in whole or in part, on human expertise were evaluated. Participants were invited to complete the trials in one of two small subsets of the full set of trials included in the core test of the main automatic system evaluation.  In a typical HASR test, each participating organization downloaded the audio data for a single trial, used their method for speaker detection on the data, and then uploaded their response to an automated system that provided the audio for the next trial.  The NIST HASR test invites voice practitioners to participate and to contribute to the further refinement of the evaluation process.  HASR evaluations have a tremendous potential to contribute to the development of the best practices and advances to the measurement science in support of voice biometrics and forensics.

An abridged version of a HASR presentation, including results, given at the 2010 Odyssey conference in Brno is available here . 

In 2012 a follow-on test confirmed the 2010 findings that combining human expertise with automatic systems to perform speaker detection poses a substantial challenge, and further research is necessary to improve the state-of-the-art in this important area.

Plans for future NIST HASR research and evaluations are ongoing.

2010 ]

*
Bookmark and Share

Lead Organizational Unit:

itl
Contact

NIST maintains a general mailing list for the Human Assisted Speaker Recognition Evaluation. Relevant evaluation information is posted to the list. If you would like to join the list or have any question for NIST related to our human assisted speaker recognition evaluation, please email us at speaker_poc@nist.gov