title: Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as
    the Key
summary: As large language models rapidly evolve to support longer context, there is a
  notable disparity in their capability to generate output at greater lengths.
  Recent study suggests that the primary cause for this imbalance may arise from
  the lack of data with long-output during alignment training. In light of this
  observation, attempts are made to re-align foundation models with data that
  fills the gap, which result in models capable of generating lengthy output when
  instructed. In this paper, we explore the impact of data-quality in tuning a
  model for long output, and the possibility of doing so from the starting points
  of human-aligned (instruct or chat) models. With careful data curation, we show
  that it possible to achieve similar performance improvement in our tuned
  models, with only a small fraction of training data instances and compute. In
  addition, we assess the generalizability of such approaches by applying our
  tuning-recipes to several models. our findings suggest that, while capacities
  for generating long output vary across different models out-of-the-box, our
  approach to tune them with high-quality data using lite compute, consistently
  yields notable improvement across all models we experimented on. We have made
  public our curated dataset for tuning long-writing capability, the
  implementations of model tuning and evaluation, as well as the fine-tuned
  models, all of which can be openly-accessed.
paper_authors._id: 67122a1054c8572e476f32a3
paper_authors.name: Yingda Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T09:54:53.711Z
paper_authors._id: 6215ca5692c0ecfba9186921
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1645595695434-6215ca5692c0ecfba9186921.png
paper_authors.fullname: Yingda Chen
paper_authors.type: user
paper_authors.user: Yingda
paper_authors._id: 67122a1054c8572e476f32a4
paper_authors.name: Xingjun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T09:54:56.405Z
paper_authors._id: 62f612e22e53c2efd33943c6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1679249371815-62f612e22e53c2efd33943c6.jpeg
paper_authors.fullname: xingjun wang
paper_authors.type: user
paper_authors.user: wangxingjun778
paper_authors._id: 67122a1054c8572e476f32a5
paper_authors.name: Jintao Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T13:03:40.531Z
paper_authors._id: 6624e6b111772517e5db916c
paper_authors.avatarUrl: /avatars/cd7f14216e63689a3ede587224e35ead.svg
paper_authors.fullname: Jintao Huang
paper_authors.type: user
paper_authors.user: feifeiyu-hjt
paper_authors._id: 67122a1054c8572e476f32a6
paper_authors.name: Yunlin Mao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T09:40:59.105Z
paper_authors._id: 64b8c458d6c468ac7525e473
paper_authors.avatarUrl: /avatars/b3000da1d230387fa0e450f098803ab4.svg
paper_authors.fullname: Lin
paper_authors.type: user
paper_authors.user: Yunxz
paper_authors._id: 67122a1054c8572e476f32a7
paper_authors.name: Daoze Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T09:54:59.082Z
paper_authors._id: 664dabaade412305ff2c984e
paper_authors.avatarUrl: /avatars/9dd55d8078f0534f2bdb8cfb84333b8e.svg
paper_authors.fullname: Zhang
paper_authors.type: user
paper_authors.user: Daoze
paper_authors._id: 67122a1054c8572e476f32a8
paper_authors.name: Yuze Zhao
paper_title: Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as
    the Key
paper_summary: As large language models rapidly evolve to support longer context, there is a
  notable disparity in their capability to generate output at greater lengths.
  Recent study suggests that the primary cause for this imbalance may arise from
  the lack of data with long-output during alignment training. In light of this
  observation, attempts are made to re-align foundation models with data that
  fills the gap, which result in models capable of generating lengthy output when
  instructed. In this paper, we explore the impact of data-quality in tuning a
  model for long output, and the possibility of doing so from the starting points
  of human-aligned (instruct or chat) models. With careful data curation, we show
  that it possible to achieve similar performance improvement in our tuned
  models, with only a small fraction of training data instances and compute. In
  addition, we assess the generalizability of such approaches by applying our
  tuning-recipes to several models. our findings suggest that, while capacities
  for generating long output vary across different models out-of-the-box, our
  approach to tune them with high-quality data using lite compute, consistently
  yields notable improvement across all models we experimented on. We have made
  public our curated dataset for tuning long-writing capability, the
  implementations of model tuning and evaluation, as well as the fine-tuned
  models, all of which can be openly-accessed.
paper_ai_keywords: large language models context output generation data alignment training long-output data re-align foundation models data-quality tuning human-aligned models instruct tuning chat tuning data curation performance improvement training data instances compute resources generalizability multi-output variation DyVo tuning-recipes long-writing capability fine-tuned models publicly-accessed dataset implementations of model tuning extensive evaluation