title: BOLT: Bootstrap Long Chain-of-Thought in Language Models without
    Distillation
summary: Large language models (LLMs), such as o1 from OpenAI, have demonstrated
  remarkable reasoning capabilities. o1 generates a long chain-of-thought
  (LongCoT) before answering a question. LongCoT allows LLMs to analyze problems,
  devise plans, reflect, and backtrack effectively. These actions empower LLM to
  solve complex problems. After the release of o1, many teams have attempted to
  replicate its LongCoT and reasoning capabilities. In terms of methods, they
  primarily rely on knowledge distillation with data from existing models with
  LongCoT capacities (e.g., OpenAI-o1, Qwen-QwQ, DeepSeek-R1-Preview), leaving
  significant uncertainties on systematically developing such reasoning
  abilities. In terms of data domains, these works focus narrowly on math while a
  few others include coding, limiting their generalizability. This paper
  introduces a novel approach to enable LLM's LongCoT capacity without
  distillation from o1-like models or expensive human annotations, where we
  bootstrap LongCoT (BOLT) from a standard instruct model. BOLT involves three
  stages: 1) LongCoT data bootstrapping with in-context learning on a standard
  instruct model; 2) LongCoT supervised finetuning; 3) online training to further
  refine LongCoT capacities. In BOLT, only a few in-context examples need to be
  constructed during the bootstrapping stage; in our experiments, we created 10
  examples, demonstrating the feasibility of this approach. We use
  Llama-3.1-70B-Instruct to bootstrap LongCoT and apply our method to various
  model scales (7B, 8B, 70B). We achieve impressive performance on a variety of
  benchmarks, Arena-Hard, MT-Bench, WildBench, ZebraLogic, MATH500, which
  evaluate diverse task-solving and reasoning capabilities.
paper_authors._id: 67a5880c886a1e223b1d57ec
paper_authors.name: Bo Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:46:17.484Z
paper_authors._id: 63e08acbf351dc0745749d56
paper_authors.avatarUrl: /avatars/8e2d5ce9db5bd8008ac2ad80f6025553.svg
paper_authors.fullname: Bo Pang
paper_authors.type: user
paper_authors.user: bpucla
paper_authors._id: 67a5880c886a1e223b1d57ed
paper_authors.name: Hanze Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:46:02.107Z
paper_authors._id: 63a3ff69f91ad3ea5703841d
paper_authors.avatarUrl: /avatars/69227c4bce01d33747c1377b6f9672db.svg
paper_authors.fullname: Hanze Dong
paper_authors.type: user
paper_authors.user: hendrydong
paper_authors._id: 67a5880c886a1e223b1d57ee
paper_authors.name: Jiacheng Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:45:54.892Z
paper_authors._id: 631983d5cb116eab31df5821
paper_authors.avatarUrl: /avatars/6a42c842a9439241ead2ace1d79fc32c.svg
paper_authors.fullname: Jiacheng Xu
paper_authors.type: user
paper_authors.user: jcxu
paper_authors._id: 67a5880c886a1e223b1d57ef
paper_authors.name: Silvio Savarese
paper_authors._id: 67a5880c886a1e223b1d57f0
paper_authors.name: Yingbo Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:45:09.887Z
paper_authors._id: 649bc93758d8b19de0c7785f
paper_authors.avatarUrl: /avatars/3ed9473aee23d99f4ee949d3705089ea.svg
paper_authors.fullname: Yingbo Zhou
paper_authors.type: user
paper_authors.user: yingbozhou
paper_authors._id: 67a5880c886a1e223b1d57f1
paper_authors.name: Caiming Xiong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:45:02.543Z
paper_authors._id: 649dbcc4e0fff1ed099dc80a
paper_authors.avatarUrl: /avatars/c87c273ca628dbcddccbf1ee19b2ce33.svg
paper_authors.fullname: Caiming Xiong
paper_authors.type: user
paper_authors.user: cxiong
paper_title: BOLT: Bootstrap Long Chain-of-Thought in Language Models without
    Distillation
paper_summary: Large language models (LLMs), such as o1 from OpenAI, have demonstrated
  remarkable reasoning capabilities. o1 generates a long chain-of-thought
  (LongCoT) before answering a question. LongCoT allows LLMs to analyze problems,
  devise plans, reflect, and backtrack effectively. These actions empower LLM to
  solve complex problems. After the release of o1, many teams have attempted to
  replicate its LongCoT and reasoning capabilities. In terms of methods, they
  primarily rely on knowledge distillation with data from existing models with
  LongCoT capacities (e.g., OpenAI-o1, Qwen-QwQ, DeepSeek-R1-Preview), leaving
  significant uncertainties on systematically developing such reasoning
  abilities. In terms of data domains, these works focus narrowly on math while a
  few others include coding, limiting their generalizability. This paper
  introduces a novel approach to enable LLM's LongCoT capacity without
  distillation from o1-like models or expensive human annotations, where we
  bootstrap LongCoT (BOLT) from a standard instruct model. BOLT involves three
  stages: 1) LongCoT data bootstrapping with in-context learning on a standard
  instruct model; 2) LongCoT supervised finetuning; 3) online training to further
  refine LongCoT capacities. In BOLT, only a few in-context examples need to be
  constructed during the bootstrapping stage; in our experiments, we created 10
  examples, demonstrating the feasibility of this approach. We use
  Llama-3.1-70B-Instruct to bootstrap LongCoT and apply our method to various
  model scales (7B, 8B, 70B). We achieve impressive performance on a variety of
  benchmarks, Arena-Hard, MT-Bench, WildBench, ZebraLogic, MATH500, which
  evaluate diverse task-solving and reasoning capabilities.
paper_ai_keywords: Large language models (LLMs) Long chain-of-thought (LongCoT) Chain-of-thought Knowledge distillation Instruct model In-context learning Supervised finetuning Online training Arena-Hard MT-Bench WildBench ZebraLogic MATH500