Overview of the TREC 2005 Robust Retrieval Track

Published

October 16, 2006

Author(s)

Ellen M. Voorhees

Abstract

The robust retrieval track explores methods for improving the consistency of retrieval technology by focusing on poorly performing topics. The retrieval task in the track is a traditional ad~hoc retrieval task where the evaluation methodology emphasizes a system's least effective topics. The 2005 edition of the track used 50 topics that had been demonstrated to be difficult on one document collection, and ran those topics on a different document collection. Relevance information from the first collection could be exploited in producing a query for the second collection, if desired. The main measure for evaluating system effectiveness is "gmap", a variant of the traditional MAP measure that uses a geometric mean rather than an arithmetic mean to average individual topic results. As in previous years, the most effective retrieval strategy was to expand queries using terms derived from additional corpora. The relative difficulty of topics differed across the two document sets.

Conference Title

Text Retrieval Conference (TREC)

Pub Type

Conferences

Download Paper

Local Download

Keywords

evaluation, search engines, TREC

Citation

Voorhees, E. (2006), Overview of the TREC 2005 Robust Retrieval Track, Text Retrieval Conference (TREC), [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=150643 (Accessed December 4, 2024)