3.5 Application-specific Capabilities

In this section, we discuss the desired capabilities in LLMs to tackle specific applications.

3.5.1 Query-focused Summarization
Benchmarks Query-focused or aspect-based summarization requires to generate summaries with
regard to a fine-grained question or an aspect category. Query-focused datasets include AQualMuse
(Kulkarni et al., 2020), QMSum (Zhong et al., 2021) and SQuALITY (Wang et al., 2022), while
Aspect-based datasets include CovidET (Zhang et al., 2023a), NEWTS (Bahrainian et al., 2022),
WikiAsp (Hayashi et al., 2021), etc.

Models Yang et al. (2023d) finds that standard fine-tuning on training data is still better in perfor-
mance compared to ChatGPT, with an average of 2 points ROUGE-1 improvement over CovidET,
NEWTS, QMSum and SQuALITY.
ChatGPT’sOne-yearAnniversary-AreOpen-Source LargeLanguageModelsCatchingup