A number of studies ( e.g. , Hildebrandt , Katz , and Lin 2004 ) have pointed out shortcomings of the original nugget scoring model , although a number of these issues have been recently addressed ( Lin and Demner-Fushman 2005a , 2006b ) .
automatically evaluating answers to definition questions
Jimmy Lin Dina Demner-Fushman
Nugget F-score has been employed as a metric in the TREC question-answering track since 2003 , to evaluate so-called definition and `` other '' questions ( Voorhees 2003 ) . A number of studies ( e.g. , Hildebrandt , Katz , and Lin 2004 ) have pointed out shortcomings of the original nugget scoring model , although a number of these issues have been recently addressed ( Lin and Demner-Fushman 2005a , 2006b ) . However , adaptation of the nugget evaluation methodology to a domain as specific as clinical medicine is an endeavor that has yet to be undertaken .
A number of studies ( e.g. , Hildebrandt , Katz , and Lin 2004 ) have pointed out shortcomings of the original nugget scoring model , although a number of these issues have been recently addressed ( @@CITATIONa , 2006b ) .