Ruler: A Model-Agnostic Method to Control Generated Length for Large
  Language Models
The instruction-following ability of large language models enables humans to
interact with AI agents in a natural way. However, when required to generate
responses of a specific length, large language models often struggle to meet
users' needs due to their inherent difficulty in accurately perceiving
numerical constraints. To explore the ability of large language models to
control the length of generated responses, we propose the Target Length
Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible
Match (FM) to evaluate the model's performance in adhering to specified
response lengths. Furthermore, we introduce a novel, model-agnostic approach
called Ruler, which employs Meta Length Tokens (MLTs) to enhance the
instruction-following ability of large language models under length-constrained
instructions. Specifically, Ruler equips LLMs with the ability to generate
responses of a specified length based on length constraints within the
instructions. Moreover, Ruler can automatically generate appropriate MLT when
length constraints are not explicitly provided, demonstrating excellent
versatility and generalization. Comprehensive experiments show the
effectiveness of Ruler across different LLMs on Target Length Generation Task,
e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In
addition, we conduct extensive ablation experiments to further substantiate the
efficacy and generalization of Ruler. Our code and data is available at
https://github.com/Geaming2002/Ruler.
66fb45a805bf0586e2813a55
Jiaming Li
claimed_verified
2024-10-01T13:10:36.240Z
6355eaf660c1b72f6269bc64
/avatars/dd176b9d6db2ac63c19c3170566a3f35.svg
Jiaming Li
user
Geaming
66fb45a805bf0586e2813a56
Lei Zhang
claimed_verified
2024-10-01T13:10:34.347Z
64c38871f9cd765462fa1a17
https://cdn-avatars.huggingface.co/v1/production/uploads/64c38871f9cd765462fa1a17/yuIlVcqeDlQVKsUF8uEl3.jpeg
Lei Zhang
user
Lemoncoke
66fb45a805bf0586e2813a57
Yunshui Li
claimed_verified
2025-06-19T15:11:40.132Z
62e670d33651180f7d334ef3
https://cdn-avatars.huggingface.co/v1/production/uploads/1661829952739-62e670d33651180f7d334ef3.jpeg
Li Yunshui
user
Wa2erGo
66fb45a805bf0586e2813a58
Ziqiang Liu
claimed_verified
2025-03-25T08:23:04.756Z
61b9b5f9fd429dff1ed3bd58
https://cdn-avatars.huggingface.co/v1/production/uploads/61b9b5f9fd429dff1ed3bd58/GrfeNU4K1zts-gmQdGbVE.jpeg
Ziqiang Liu
user
icoderzqliu
66fb45a805bf0586e2813a59
yuelin bai
66fb45a805bf0586e2813a5a
Run Luo
66fb45a805bf0586e2813a5b
Longze Chen
claimed_verified
2024-10-31T09:55:30.740Z
64c7b4d1c547ed5243c07b6c
https://cdn-avatars.huggingface.co/v1/production/uploads/64c7b4d1c547ed5243c07b6c/h96CLBj6dcm01soK2UJzr.jpeg
Longze Chen
user
lzchen2001
66fb45a805bf0586e2813a5c
Min Yang
Ruler: A Model-Agnostic Method to Control Generated Length for Large
  Language Models
The instruction-following ability of large language models enables humans to
interact with AI agents in a natural way. However, when required to generate
responses of a specific length, large language models often struggle to meet
users' needs due to their inherent difficulty in accurately perceiving
numerical constraints. To explore the ability of large language models to
control the length of generated responses, we propose the Target Length
Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible
Match (FM) to evaluate the model's performance in adhering to specified
response lengths. Furthermore, we introduce a novel, model-agnostic approach
called Ruler, which employs Meta Length Tokens (MLTs) to enhance the
instruction-following ability of large language models under length-constrained
instructions. Specifically, Ruler equips LLMs with the ability to generate
responses of a specified length based on length constraints within the
instructions. Moreover, Ruler can automatically generate appropriate MLT when
length constraints are not explicitly provided, demonstrating excellent
versatility and generalization. Comprehensive experiments show the
effectiveness of Ruler across different LLMs on Target Length Generation Task,
e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In
addition, we conduct extensive ablation experiments to further substantiate the
efficacy and generalization of Ruler. Our code and data is available at
https://github.com/Geaming2002/Ruler.
A novel approach, Ruler, uses Meta Length Tokens to improve large language models' adherence to specified response lengths in generated text.
large language models Target Length Generation Task Precise Match Flexible Match Ruler Meta Length Tokens