title: Token Assorted: Mixing Latent and Text Tokens for Improved Language
    Model Reasoning
summary: Large Language Models (LLMs) excel at reasoning and planning when trained on
  chainof-thought (CoT) data, where the step-by-step thought process is
  explicitly outlined by text tokens. However, this results in lengthy inputs
  where many words support textual coherence rather than core reasoning
  information, and processing these inputs consumes substantial computation
  resources. In this work, we propose a hybrid representation of the reasoning
  process, where we partially abstract away the initial reasoning steps using
  latent discrete tokens generated by VQ-VAE, significantly reducing the length
  of reasoning traces. We explore the use of latent trace abstractions in two
  scenarios: 1) training the model from scratch for the Keys-Finding Maze
  problem, 2) fine-tuning LLMs on this hybrid data with an extended vocabulary
  including unseen latent tokens, for both logical and mathematical reasoning
  problems. To facilitate effective learning, we introduce a simple training
  procedure that randomly mixes latent and text tokens, which enables fast
  adaptation to new latent tokens. Our approach consistently outperforms the
  baselines methods in various benchmarks.
paper_authors._id: 67a448b69ca42c642a723a7d
paper_authors.name: DiJia Su
paper_authors._id: 67a448b69ca42c642a723a7e
paper_authors.name: Hanlin Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-06T16:10:24.407Z
paper_authors._id: 6467bc59b990713c50339d2d
paper_authors.avatarUrl: /avatars/064aba45e37040f7b1de8f76169f5174.svg
paper_authors.fullname: Hanlin Zhu
paper_authors.type: user
paper_authors.user: hanlinzhu
paper_authors._id: 67a448b69ca42c642a723a7f
paper_authors.name: Yingchen Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-06T16:09:40.111Z
paper_authors._id: 6481333f8c6a3b8f11fc4114
paper_authors.avatarUrl: /avatars/15c194296506b32e3f218530382c9f78.svg
paper_authors.fullname: Yingchen Xu
paper_authors.type: user
paper_authors.user: xuyingchen
paper_authors._id: 67a448b69ca42c642a723a80
paper_authors.name: Jiantao Jiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-06T16:09:29.786Z
paper_authors._id: 653b306986b88947d5cacfa4
paper_authors.avatarUrl: /avatars/21ebd4daf35ec67c7d5f9b0a53628b00.svg
paper_authors.fullname: Jiantao Jiao
paper_authors.type: user
paper_authors.user: nexus-jt-llm
paper_authors._id: 67a448b69ca42c642a723a81
paper_authors.name: Yuandong Tian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-06T16:09:19.670Z
paper_authors._id: 6344cf73ee1504dbcd5bdfe7
paper_authors.avatarUrl: /avatars/6dd2bf1f9c5679e5c8c85d62c9836aac.svg
paper_authors.fullname: Yuandong Tian
paper_authors.type: user
paper_authors.user: tydsh
paper_authors._id: 67a448b69ca42c642a723a82
paper_authors.name: Qinqing Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-06T16:09:11.277Z
paper_authors._id: 64d27579dafee18faf9308ac
paper_authors.avatarUrl: /avatars/8914a47244017c3541d3d5ac5b2d0372.svg
paper_authors.fullname: Qinqing Zheng
paper_authors.type: user
paper_authors.user: goodsleep
paper_title: Token Assorted: Mixing Latent and Text Tokens for Improved Language
    Model Reasoning
paper_summary: Large Language Models (LLMs) excel at reasoning and planning when trained on
  chainof-thought (CoT) data, where the step-by-step thought process is
  explicitly outlined by text tokens. However, this results in lengthy inputs
  where many words support textual coherence rather than core reasoning
  information, and processing these inputs consumes substantial computation
  resources. In this work, we propose a hybrid representation of the reasoning
  process, where we partially abstract away the initial reasoning steps using
  latent discrete tokens generated by VQ-VAE, significantly reducing the length
  of reasoning traces. We explore the use of latent trace abstractions in two
  scenarios: 1) training the model from scratch for the Keys-Finding Maze
  problem, 2) fine-tuning LLMs on this hybrid data with an extended vocabulary
  including unseen latent tokens, for both logical and mathematical reasoning
  problems. To facilitate effective learning, we introduce a simple training
  procedure that randomly mixes latent and text tokens, which enables fast
  adaptation to new latent tokens. Our approach consistently outperforms the
  baselines methods in various benchmarks.
paper_ai_keywords: VQ-VAE latent discrete tokens hybrid tokens Keys-Finding Maze problem latent trace abstractions parameter-efficient fine-tuning