title: PositionID: LLMs can Control Lengths, Copy and Paste with Explicit
    Positional Awareness
summary: Large Language Models (LLMs) demonstrate impressive capabilities across
  various domains, including role-playing, creative writing, mathematical
  reasoning, and coding. Despite these advancements, LLMs still encounter
  challenges with length control, frequently failing to adhere to specific length
  constraints due to their token-level operations and insufficient training on
  data with strict length limitations. We identify this issue as stemming from a
  lack of positional awareness and propose novel approaches--PositionID Prompting
  and PositionID Fine-Tuning--to address it. These methods enhance the model's
  ability to continuously monitor and manage text length during generation.
  Additionally, we introduce PositionID CP Prompting to enable LLMs to perform
  copy and paste operations accurately. Furthermore, we develop two benchmarks
  for evaluating length control and copy-paste abilities. Our experiments
  demonstrate that our methods significantly improve the model's adherence to
  length constraints and copy-paste accuracy without compromising response
  quality.
paper_authors._id: 6707d46de2deb345cb3bda89
paper_authors.name: Zekun Wang
paper_authors._id: 6707d46de2deb345cb3bda8a
paper_authors.name: Feiyu Duan
paper_authors._id: 6707d46de2deb345cb3bda8b
paper_authors.name: Yibo Zhang
paper_authors._id: 6707d46de2deb345cb3bda8c
paper_authors.name: Wangchunshu Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T11:16:28.376Z
paper_authors._id: 628c8598ef14f971b698107f
paper_authors.avatarUrl: /avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Wangchunshu
paper_authors._id: 6707d46de2deb345cb3bda8d
paper_authors.name: Ke Xu
paper_authors._id: 6707d46de2deb345cb3bda8e
paper_authors.name: Wenhao Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:16:29.003Z
paper_authors._id: 65bb11cb00a03997849e9e85
paper_authors.avatarUrl: /avatars/17022b0254192a837f4fe00d84389cda.svg
paper_authors.fullname: Wenhao Huang
paper_authors.type: user
paper_authors.user: StephenHuang
paper_authors._id: 6707d46de2deb345cb3bda8f
paper_authors.name: Jie Fu
paper_title: PositionID: LLMs can Control Lengths, Copy and Paste with Explicit
    Positional Awareness
paper_summary: Large Language Models (LLMs) demonstrate impressive capabilities across
  various domains, including role-playing, creative writing, mathematical
  reasoning, and coding. Despite these advancements, LLMs still encounter
  challenges with length control, frequently failing to adhere to specific length
  constraints due to their token-level operations and insufficient training on
  data with strict length limitations. We identify this issue as stemming from a
  lack of positional awareness and propose novel approaches--PositionID Prompting
  and PositionID Fine-Tuning--to address it. These methods enhance the model's
  ability to continuously monitor and manage text length during generation.
  Additionally, we introduce PositionID CP Prompting to enable LLMs to perform
  copy and paste operations accurately. Furthermore, we develop two benchmarks
  for evaluating length control and copy-paste abilities. Our experiments
  demonstrate that our methods significantly improve the model's adherence to
  length constraints and copy-paste accuracy without compromising response
  quality.
paper_ai_keywords: LLMs Large Language Models token-level operations positional awareness PositionID Prompting PositionID Fine-Tuning PositionID CP Prompting length control benchmarks