Nugget F-score has been employed as a metric in the TREC question-answering track since 2003 , to evaluate so-called definition and `` other '' questions ( Voorhees 2003 ) .
overview of the trec
Ellen M Voorhees
Recently , there is a growing consensus that an evaluation methodology based on the notion of `` information nuggets '' may provide an appropriate framework for assessing the quality of answers to complex questions . Nugget F-score has been employed as a metric in the TREC question-answering track since 2003 , to evaluate so-called definition and `` other '' questions ( Voorhees 2003 ) . A number of studies ( e.g. , Hildebrandt , Katz , and Lin 2004 ) have pointed out shortcomings of the original nugget scoring model , although a number of these issues have been recently addressed ( Lin and Demner-Fushman 2005a , 2006b ) .
Nugget F-score has been employed as a metric in the TREC question-answering track since 2003 , to evaluate so-called definition and `` other '' questions ( @@CITATION ) .