title: o1-Coder: an o1 Replication for Coding
summary: The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1
  model with a focus on coding tasks. It integrates reinforcement learning (RL)
  and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking
  capabilities. The framework includes training a Test Case Generator (TCG) for
  standardized code testing, using MCTS to generate code data with reasoning
  processes, and iteratively fine-tuning the policy model to initially produce
  pseudocode, followed by the generation of the full code. The report also
  addresses the opportunities and challenges in deploying o1-like models in
  real-world applications, suggesting transitioning to the System-2 paradigm and
  highlighting the imperative for environment state updates. Updated model
  progress and experimental results will be reported in subsequent versions. All
  source code, curated datasets, as well as the derived models will be disclosed
  at https://github.com/ADaM-BJTU/O1-CODER .
paper_authors._id: 674e99013209f1fbb76bbec6
paper_authors.name: Yuxiang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-05T09:49:52.207Z
paper_authors._id: 645b4a2978730bcc103dfe4d
paper_authors.avatarUrl: /avatars/de544de899897fd0a83506ff287123bc.svg
paper_authors.fullname: Yuxiang Zhang
paper_authors.type: user
paper_authors.user: TokerZ
paper_authors._id: 674e99013209f1fbb76bbec7
paper_authors.name: Shangxi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T15:16:33.713Z
paper_authors._id: 665ace40ddba0f825f3a4941
paper_authors.avatarUrl: /avatars/ed7e209e14b5ae495aa7f4baec980b38.svg
paper_authors.fullname: Shangxi Wu
paper_authors.type: user
paper_authors.user: KirinNg
paper_authors._id: 674e99013209f1fbb76bbec8
paper_authors.name: Yuqi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T15:16:27.807Z
paper_authors._id: 65b5b586d2ae98fd67c76d6b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RR_mNkUAG4zP902WLgeYv.jpeg
paper_authors.fullname: yuqi yang
paper_authors.type: user
paper_authors.user: tzteyang
paper_authors._id: 674e99013209f1fbb76bbec9
paper_authors.name: Jiangming Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T15:16:20.077Z
paper_authors._id: 6540f059733c1ce6a6fcca88
paper_authors.avatarUrl: /avatars/b8fa6b383caeb1fe79e163bac90d5cba.svg
paper_authors.fullname: ShuJiangMing
paper_authors.type: user
paper_authors.user: SanJiaoMao
paper_authors._id: 674e99013209f1fbb76bbeca
paper_authors.name: Jinlin Xiao
paper_authors._id: 674e99013209f1fbb76bbecb
paper_authors.name: Chao Kong
paper_authors._id: 674e99013209f1fbb76bbecc
paper_authors.name: Jitao Sang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:46:19.856Z
paper_authors._id: 6494457c6339264dd78bcb95
paper_authors.avatarUrl: /avatars/d87842251f1a43f50cc827f0e2a995ee.svg
paper_authors.fullname: sdzy
paper_authors.type: user
paper_authors.user: sdzy
paper_title: o1-Coder: an o1 Replication for Coding
paper_summary: The technical report introduces O1-CODER, an attempt to replicate OpenAI's o1
  model with a focus on coding tasks. It integrates reinforcement learning (RL)
  and Monte Carlo Tree Search (MCTS) to enhance the model's System-2 thinking
  capabilities. The framework includes training a Test Case Generator (TCG) for
  standardized code testing, using MCTS to generate code data with reasoning
  processes, and iteratively fine-tuning the policy model to initially produce
  pseudocode, followed by the generation of the full code. The report also
  addresses the opportunities and challenges in deploying o1-like models in
  real-world applications, suggesting transitioning to the System-2 paradigm and
  highlighting the imperative for environment state updates. Updated model
  progress and experimental results will be reported in subsequent versions. All
  source code, curated datasets, as well as the derived models will be disclosed
  at https://github.com/ADaM-BJTU/O1-CODER .
paper_ai_keywords: reinforcement learning (RL) Monte Carlo Tree Search (MCTS) Test Case Generator (TCG) pseudocode System-2 thinking capabilities System-2 paradigm environment state updates