title: Supervised Knowledge Makes Large Language Models Better In-context
    Learners
summary: Large Language Models (LLMs) exhibit emerging in-context learning abilities
  through prompt engineering. The recent progress in large-scale generative
  models has further expanded their use in real-world language applications.
  However, the critical challenge of improving the generalizability and
  factuality of LLMs in natural language understanding and question answering
  remains under-explored. While previous in-context learning research has focused
  on enhancing models to adhere to users' specific instructions and quality
  expectations, and to avoid undesired outputs, little to no work has explored
  the use of task-Specific fine-tuned Language Models (SLMs) to improve LLMs'
  in-context learning during the inference stage. Our primary contribution is the
  establishment of a simple yet effective framework that enhances the reliability
  of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs
  benefit from discriminative models, and 3) minimizes hallucinations in
  generative tasks. Using our proposed plug-in method, enhanced versions of Llama
  2 and ChatGPT surpass their original versions regarding generalizability and
  factuality. We offer a comprehensive suite of resources, including 16 curated
  datasets, prompts, model checkpoints, and LLM outputs across 9 distinct tasks.
  Our empirical analysis sheds light on the advantages of incorporating
  discriminative models into LLMs and highlights the potential of our methodology
  in fostering more reliable LLMs.
paper_authors._id: 658b97a9135580745c460614
paper_authors.name: Linyi Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-29T08:59:38.465Z
paper_authors._id: 64895683f534abe18eec264b
paper_authors.avatarUrl: /avatars/73cc9e6db6db86793787750776b57c63.svg
paper_authors.fullname: Linyi Yang
paper_authors.type: user
paper_authors.user: linyiyang2023
paper_authors._id: 658b97a9135580745c460615
paper_authors.name: Shuibai Zhang
paper_authors._id: 658b97a9135580745c460616
paper_authors.name: Zhuohao Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T07:34:51.040Z
paper_authors._id: 61e24808b31e7cc38eb84d37
paper_authors.avatarUrl: /avatars/65fbea940fad211462ecc5ad725e0c28.svg
paper_authors.fullname: Zhuohao Yu
paper_authors.type: user
paper_authors.user: zhuohaoyu
paper_authors._id: 658b97a9135580745c460617
paper_authors.name: Guangsheng Bao
paper_authors._id: 658b97a9135580745c460618
paper_authors.name: Yidong Wang
paper_authors._id: 658b97a9135580745c460619
paper_authors.name: Jindong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T08:02:42.285Z
paper_authors._id: 6204cc0d522e40b4a18d86e2
paper_authors.avatarUrl: /avatars/18daf2de5671e711dc745388dd60569d.svg
paper_authors.fullname: Jindong Wang
paper_authors.type: user
paper_authors.user: jindongwang
paper_authors._id: 658b97a9135580745c46061a
paper_authors.name: Ruochen Xu
paper_authors._id: 658b97a9135580745c46061b
paper_authors.name: Wei Ye
paper_authors._id: 658b97a9135580745c46061c
paper_authors.name: Xing Xie
paper_authors._id: 658b97a9135580745c46061d
paper_authors.name: Weizhu Chen
paper_authors._id: 658b97a9135580745c46061e
paper_authors.name: Yue Zhang
paper_title: Supervised Knowledge Makes Large Language Models Better In-context
    Learners
paper_summary: Large Language Models (LLMs) exhibit emerging in-context learning abilities
  through prompt engineering. The recent progress in large-scale generative
  models has further expanded their use in real-world language applications.
  However, the critical challenge of improving the generalizability and
  factuality of LLMs in natural language understanding and question answering
  remains under-explored. While previous in-context learning research has focused
  on enhancing models to adhere to users' specific instructions and quality
  expectations, and to avoid undesired outputs, little to no work has explored
  the use of task-Specific fine-tuned Language Models (SLMs) to improve LLMs'
  in-context learning during the inference stage. Our primary contribution is the
  establishment of a simple yet effective framework that enhances the reliability
  of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs
  benefit from discriminative models, and 3) minimizes hallucinations in
  generative tasks. Using our proposed plug-in method, enhanced versions of Llama
  2 and ChatGPT surpass their original versions regarding generalizability and
  factuality. We offer a comprehensive suite of resources, including 16 curated
  datasets, prompts, model checkpoints, and LLM outputs across 9 distinct tasks.
  Our empirical analysis sheds light on the advantages of incorporating
  discriminative models into LLMs and highlights the potential of our methodology
  in fostering more reliable LLMs.
paper_ai_summary: A framework using task-specific fine-tuning improves large language models' generalizability and factuality during inference by leveraging discriminative models.
paper_ai_keywords: Large Language Models in-context learning prompt engineering generative models task-Specific fine-tuned Language Models discriminative models out-of-distribution data hallucinations generalizability factuality