title: Ruler: A Model-Agnostic Method to Control Generated Length for Large
    Language Models
summary: The instruction-following ability of large language models enables humans to
  interact with AI agents in a natural way. However, when required to generate
  responses of a specific length, large language models often struggle to meet
  users' needs due to their inherent difficulty in accurately perceiving
  numerical constraints. To explore the ability of large language models to
  control the length of generated responses, we propose the Target Length
  Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible
  Match (FM) to evaluate the model's performance in adhering to specified
  response lengths. Furthermore, we introduce a novel, model-agnostic approach
  called Ruler, which employs Meta Length Tokens (MLTs) to enhance the
  instruction-following ability of large language models under length-constrained
  instructions. Specifically, Ruler equips LLMs with the ability to generate
  responses of a specified length based on length constraints within the
  instructions. Moreover, Ruler can automatically generate appropriate MLT when
  length constraints are not explicitly provided, demonstrating excellent
  versatility and generalization. Comprehensive experiments show the
  effectiveness of Ruler across different LLMs on Target Length Generation Task,
  e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In
  addition, we conduct extensive ablation experiments to further substantiate the
  efficacy and generalization of Ruler. Our code and data is available at
  https://github.com/Geaming2002/Ruler.
paper_authors._id: 66fb45a805bf0586e2813a55
paper_authors.name: Jiaming Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-01T13:10:36.240Z
paper_authors._id: 6355eaf660c1b72f6269bc64
paper_authors.avatarUrl: /avatars/dd176b9d6db2ac63c19c3170566a3f35.svg
paper_authors.fullname: Jiaming Li
paper_authors.type: user
paper_authors.user: Geaming
paper_authors._id: 66fb45a805bf0586e2813a56
paper_authors.name: Lei Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-01T13:10:34.347Z
paper_authors._id: 64c38871f9cd765462fa1a17
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c38871f9cd765462fa1a17/yuIlVcqeDlQVKsUF8uEl3.jpeg
paper_authors.fullname: Lei Zhang
paper_authors.type: user
paper_authors.user: Lemoncoke
paper_authors._id: 66fb45a805bf0586e2813a57
paper_authors.name: Yunshui Li
paper_authors._id: 66fb45a805bf0586e2813a58
paper_authors.name: Ziqiang Liu
paper_authors._id: 66fb45a805bf0586e2813a59
paper_authors.name: yuelin bai
paper_authors._id: 66fb45a805bf0586e2813a5a
paper_authors.name: Run Luo
paper_authors._id: 66fb45a805bf0586e2813a5b
paper_authors.name: Longze Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-31T09:55:30.740Z
paper_authors._id: 64c7b4d1c547ed5243c07b6c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c7b4d1c547ed5243c07b6c/h96CLBj6dcm01soK2UJzr.jpeg
paper_authors.fullname: Longze Chen
paper_authors.type: user
paper_authors.user: lzchen2001
paper_authors._id: 66fb45a805bf0586e2813a5c
paper_authors.name: Min Yang
paper_title: Ruler: A Model-Agnostic Method to Control Generated Length for Large
    Language Models
paper_summary: The instruction-following ability of large language models enables humans to
  interact with AI agents in a natural way. However, when required to generate
  responses of a specific length, large language models often struggle to meet
  users' needs due to their inherent difficulty in accurately perceiving
  numerical constraints. To explore the ability of large language models to
  control the length of generated responses, we propose the Target Length
  Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible
  Match (FM) to evaluate the model's performance in adhering to specified
  response lengths. Furthermore, we introduce a novel, model-agnostic approach
  called Ruler, which employs Meta Length Tokens (MLTs) to enhance the
  instruction-following ability of large language models under length-constrained
  instructions. Specifically, Ruler equips LLMs with the ability to generate
  responses of a specified length based on length constraints within the
  instructions. Moreover, Ruler can automatically generate appropriate MLT when
  length constraints are not explicitly provided, demonstrating excellent
  versatility and generalization. Comprehensive experiments show the
  effectiveness of Ruler across different LLMs on Target Length Generation Task,
  e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In
  addition, we conduct extensive ablation experiments to further substantiate the
  efficacy and generalization of Ruler. Our code and data is available at
  https://github.com/Geaming2002/Ruler.
paper_ai_keywords: Target Length Generation Task (TLG) Precise Match (PM) Flexible Match (FM) Meta Length Tokens (MLTs) instruction-following ability length-constrained instructions All Level