Searching for Answers in a Pandemic: An Overview of TREC-COVID

Published

September 1, 2021

Author(s)

Ellen M. Voorhees, Ian Soboroff, Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Lucy L. Wang, William Hersh

Abstract

We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July, 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.

Citation

Journal of Biomedical Informatics

Volume

121

Pub Type

Journals

Download Paper

DOI Link

Keywords

COVID, information retrieval, test collections, TREC

Citation

Voorhees, E. , Soboroff, I. , Roberts, K. , Alam, T. , Bedrick, S. , Demner-Fushman, D. , Lo, K. , Wang, L. and Hersh, W. (2021), Searching for Answers in a Pandemic: An Overview of TREC-COVID, Journal of Biomedical Informatics, [online], https://doi.org/10.1016/j.jbi.2021.103865 (Accessed November 23, 2024)