4.2 Benchmarks for Specific Downstream Tasks
Other than benchmarks for general tasks, there exist benchmarks specifically designed for certain
downstream tasks.

Question-answering benchmarks have become a fundamental component in the assessment of
LLMs and their overall performance. MultiMedQA [168] is a medical QA benchmark that focuses
on medical examinations, medical research, and consumer healthcare questions. It consists of seven
datasets related to medical QA, including six existing datasets and one new dataset. The goal of
this benchmark is to evaluate the performance of LLMs in terms of clinical knowledge and QA
abilities. To assess the ability of LLMs in dynamic QA about current world knowledge, Vu et al.
[186] introduced FRESHQA. By incorporating relevant and current information retrieved from
search engines into prompts, there is a significant enhancement in the performance of LLMs on

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.
ASurveyonEvaluationofLargeLanguageModels