The 2022 NIST Language Recognition Evaluation

Published

February 28, 2023

Author(s)

Yooyoung Lee, Craig Greenberg, Asad Butt, Eliot Godard, Elliot Singer, Trang Nguyen, Lisa Mason, Douglas Reynolds

Abstract

In 2022, the U.S. National Institute of Standards and Technology (NIST) conducted a Language Recognition Evaluation (LRE), which was the latest in an ongoing series of language detection evaluations administered by NIST since 1996. The LREs measure how well state-of-the-art technology can automatically detect a target language being spoken in an audio segment. LRE22 primarily focused on low resource languages, and the task for LRE22 was language detection with 14 target languages, namely, Afrikaans, Tunisian Arabic, Algerian Arabic, Libyan Arabic, South African English, Indian-accented South African English, North African French, Ndebele, Oromo, Tigrinya, Tsonga, Venda, Xhosa, and Zulu. Unlike the previous LRE conducted in 2017, language clusters were not utilized in LRE22 and the the amount of speech in the audio segments were sampled from a uniform distribution. A total of 16 teams from 21 academic and industrial organizations in 13 different countries participated in LRE22 and made 65 valid system submissions. This paper presents an overview of LRE22 and an analysis of system performance under different evaluation conditions.

Proceedings Title

INTERSPEECH 2023

Conference Dates

August 20-24, 2023

Conference Location

Dublin, IE

Pub Type

Conferences

Download Paper

Local Download

Keywords

human language technology, NIST LRE, language recognition, language detection, speech technology performance evaluation

Citation

Lee, Y. , Greenberg, C. , Butt, A. , Godard, E. , Singer, E. , Nguyen, T. , Mason, L. and Reynolds, D. (2023), The 2022 NIST Language Recognition Evaluation, INTERSPEECH 2023, Dublin, IE, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=936367 (Accessed March 14, 2025)