title: Beyond In-Context Learning: Aligning Long-form Generation of Large
    Language Models via Task-Inherent Attribute Guidelines
summary: In-context learning (ICL) is an important yet not fully understood ability of
  pre-trained large language models (LLMs). It can greatly enhance task
  performance using a few examples, termed demonstrations, without fine-tuning.
  Although effective in question answering, ICL often underperforms in long-form
  generation tasks such as summarization. Under appropriately realistic
  assumptions, we empirically and theoretically show that ICL demonstrations
  alone are insufficient to teach LLMs the task language and format distributions
  for generation. We argue for explicit exposure to the task distributions and
  hypothesize that defining them by prompting enhances model performance. To this
  end, we present LongGuide, which efficiently generates two parallel streams of
  guidelines capturing task language and format properties: (i) Metric Guidelines
  (MGs) that instruct models to optimize self-evaluated metrics; and (ii) Output
  Constraint Guidelines (OCGs) that constrain generation at both token and
  sentence levels. LongGuide automatically selects the best combination of
  guidelines, improving both strong open- and closed-source LLMs by over 5% in
  both zero- and few-shot settings. We show that LongGuide is generalizable,
  learnable by weak models to enhance strong ones, and integrates synergistically
  with automatic prompt optimizers.
paper_authors._id: 68406f484024b3c937eebb9b
paper_authors.name: Do Xuan Long
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:27:41.485Z
paper_authors._id: 63a9a0d13453852ef53c0b37
paper_authors.avatarUrl: /avatars/411c4ffc2a3e89047a23c6e7442f6ed5.svg
paper_authors.fullname: Do Xuan Long
paper_authors.type: user
paper_authors.user: dxlong2000
paper_authors._id: 68406f484024b3c937eebb9c
paper_authors.name: Duong Ngoc Yen
paper_authors._id: 68406f484024b3c937eebb9d
paper_authors.name: Do Xuan Trong
paper_authors._id: 68406f484024b3c937eebb9e
paper_authors.name: Luu Anh Tuan
paper_authors._id: 68406f484024b3c937eebb9f
paper_authors.name: Kenji Kawaguchi
paper_authors._id: 68406f484024b3c937eebba0
paper_authors.name: Shafiq Joty
paper_authors._id: 68406f484024b3c937eebba1
paper_authors.name: Min-Yen Kan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:34:41.342Z
paper_authors._id: 628f137053028c04a3b750e4
paper_authors.avatarUrl: /avatars/85d174cc041ab91e724bb9d9d4e46c88.svg
paper_authors.fullname: Min-Yen Kan
paper_authors.type: user
paper_authors.user: knmnyn
paper_authors._id: 68406f484024b3c937eebba2
paper_authors.name: Nancy F. Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:34:35.789Z
paper_authors._id: 6774ab7a1f7f8da50570ec29
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6774ab7a1f7f8da50570ec29/5w71bqq_w3QxWbb1FLyto.jpeg
paper_authors.fullname: Nancy F. Chen
paper_authors.type: user
paper_authors.user: nancy-f-chen
paper_title: Beyond In-Context Learning: Aligning Long-form Generation of Large
    Language Models via Task-Inherent Attribute Guidelines
paper_summary: In-context learning (ICL) is an important yet not fully understood ability of
  pre-trained large language models (LLMs). It can greatly enhance task
  performance using a few examples, termed demonstrations, without fine-tuning.
  Although effective in question answering, ICL often underperforms in long-form
  generation tasks such as summarization. Under appropriately realistic
  assumptions, we empirically and theoretically show that ICL demonstrations
  alone are insufficient to teach LLMs the task language and format distributions
  for generation. We argue for explicit exposure to the task distributions and
  hypothesize that defining them by prompting enhances model performance. To this
  end, we present LongGuide, which efficiently generates two parallel streams of
  guidelines capturing task language and format properties: (i) Metric Guidelines
  (MGs) that instruct models to optimize self-evaluated metrics; and (ii) Output
  Constraint Guidelines (OCGs) that constrain generation at both token and
  sentence levels. LongGuide automatically selects the best combination of
  guidelines, improving both strong open- and closed-source LLMs by over 5% in
  both zero- and few-shot settings. We show that LongGuide is generalizable,
  learnable by weak models to enhance strong ones, and integrates synergistically
  with automatic prompt optimizers.
paper_ai_summary: LongGuide enhances pre-trained LLMs in long-form generation tasks by providing metric and output constraint guidelines, improving model performance in zero- and few-shot settings.
paper_ai_keywords: in-context learning pre-trained large language models task performance demonstrations long-form generation summarization ICL task language format distributions Metric Guidelines Output Constraint Guidelines zero-shot few-shot