title: DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for
    Reinforcement Learning and Monte-Carlo Tree Search
summary: We introduce DeepSeek-Prover-V1.5, an open-source language model designed for
  theorem proving in Lean 4, which enhances DeepSeek-Prover-V1 by optimizing both
  training and inference processes. Pre-trained on DeepSeekMath-Base with
  specialization in formal mathematical languages, the model undergoes supervised
  fine-tuning using an enhanced formal theorem proving dataset derived from
  DeepSeek-Prover-V1. Further refinement is achieved through reinforcement
  learning from proof assistant feedback (RLPAF). Beyond the single-pass
  whole-proof generation approach of DeepSeek-Prover-V1, we propose RMaxTS, a
  variant of Monte-Carlo tree search that employs an intrinsic-reward-driven
  exploration strategy to generate diverse proof paths. DeepSeek-Prover-V1.5
  demonstrates significant improvements over DeepSeek-Prover-V1, achieving new
  state-of-the-art results on the test set of the high school level miniF2F
  benchmark (63.5%) and the undergraduate level ProofNet benchmark (25.3%).
paper_authors._id: 66bebba0c55655c7150bb379
paper_authors.name: Huajian Xin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:34:26.149Z
paper_authors._id: 6532a060a78e70d19c669103
paper_authors.avatarUrl: /avatars/3cc9309b0e31da0fb83f1c3ef87dbe9f.svg
paper_authors.fullname: HuajianXin
paper_authors.type: user
paper_authors.user: HuajianXin
paper_authors._id: 66bebba0c55655c7150bb37a
paper_authors.name: Z. Z. Ren
paper_authors._id: 66bebba0c55655c7150bb37b
paper_authors.name: Junxiao Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:34:35.317Z
paper_authors._id: 6565a2dd131d13ccc5d8cb12
paper_authors.avatarUrl: /avatars/f5c5441ba74791b64c9740911f952bac.svg
paper_authors.fullname: Junxiao Song
paper_authors.type: user
paper_authors.user: haha-point
paper_authors._id: 66bebba0c55655c7150bb37c
paper_authors.name: Zhihong Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:34:41.552Z
paper_authors._id: 65db64f8b62d242ed8711701
paper_authors.avatarUrl: /avatars/753e9f980eb6786c6b53b2f1becbf745.svg
paper_authors.fullname: Zhihong Shao
paper_authors.type: user
paper_authors.user: ZhihongShao
paper_authors._id: 66bebba0c55655c7150bb37d
paper_authors.name: Wanjia Zhao
paper_authors._id: 66bebba0c55655c7150bb37e
paper_authors.name: Haocheng Wang
paper_authors._id: 66bebba0c55655c7150bb37f
paper_authors.name: Bo Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T08:19:20.542Z
paper_authors._id: 635e3a76106f984574c36409
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
paper_authors.fullname: Bo Liu
paper_authors.type: user
paper_authors.user: Benjamin-eecs
paper_authors._id: 66bebba0c55655c7150bb380
paper_authors.name: Liyue Zhang
paper_authors._id: 66bebba0c55655c7150bb381
paper_authors.name: Xuan Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:35:20.244Z
paper_authors._id: 65965cd1c4941e82831cb659
paper_authors.avatarUrl: /avatars/1aefb9cf4a124cd82f1db58d3a004a0e.svg
paper_authors.fullname: Xuan Lu
paper_authors.type: user
paper_authors.user: lx09
paper_authors._id: 66bebba0c55655c7150bb382
paper_authors.name: Qiushi Du
paper_authors._id: 66bebba0c55655c7150bb383
paper_authors.name: Wenjun Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:35:35.828Z
paper_authors._id: 663a24590966eef8680e70ed
paper_authors.avatarUrl: /avatars/32cd23e8f8405d62e245723510b2e36a.svg
paper_authors.fullname: wenjun gao
paper_authors.type: user
paper_authors.user: wjga001
paper_authors._id: 66bebba0c55655c7150bb384
paper_authors.name: Qihao Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:35:46.380Z
paper_authors._id: 63cd76b4374057a338e8e703
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63cd76b4374057a338e8e703/i4Qk5-0aYx3oRhC8b50aJ.jpeg
paper_authors.fullname: zhuqihao
paper_authors.type: user
paper_authors.user: zqh11
paper_authors._id: 66bebba0c55655c7150bb385
paper_authors.name: Dejian Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:35:55.805Z
paper_authors._id: 6225bb44c6e650de3a65dbaa
paper_authors.avatarUrl: /avatars/99c99ced2461978df572c27c1b3a4904.svg
paper_authors.fullname: DejianYang
paper_authors.type: user
paper_authors.user: DejianYang
paper_authors._id: 66bebba0c55655c7150bb386
paper_authors.name: Zhibin Gou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:36:06.134Z
paper_authors._id: 62dcf5d4169bd1d2ef2ca724
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62dcf5d4169bd1d2ef2ca724/oRFFmJDJTLYtPRVPCweQ_.jpeg
paper_authors.fullname: Zhibin Gou
paper_authors.type: user
paper_authors.user: zubingou
paper_authors._id: 66bebba0c55655c7150bb387
paper_authors.name: Z. F. Wu
paper_authors._id: 66bebba0c55655c7150bb388
paper_authors.name: Fuli Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T08:36:54.274Z
paper_authors._id: 6538815d1bdb3c40db94fbfa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6538815d1bdb3c40db94fbfa/id7aSY8JUgKK2agKWLERt.jpeg
paper_authors.fullname: Fuli Luo
paper_authors.type: user
paper_authors.user: luofuli
paper_authors._id: 66bebba0c55655c7150bb389
paper_authors.name: Chong Ruan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-17T05:38:46.449Z
paper_authors._id: 6398203609f12714ed1935c2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6398203609f12714ed1935c2/uXgl0LgKnFYjq1Wz39-a6.jpeg
paper_authors.fullname: Chong Ruan
paper_authors.type: user
paper_authors.user: Chester111
paper_title: DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for
    Reinforcement Learning and Monte-Carlo Tree Search
paper_summary: We introduce DeepSeek-Prover-V1.5, an open-source language model designed for
  theorem proving in Lean 4, which enhances DeepSeek-Prover-V1 by optimizing both
  training and inference processes. Pre-trained on DeepSeekMath-Base with
  specialization in formal mathematical languages, the model undergoes supervised
  fine-tuning using an enhanced formal theorem proving dataset derived from
  DeepSeek-Prover-V1. Further refinement is achieved through reinforcement
  learning from proof assistant feedback (RLPAF). Beyond the single-pass
  whole-proof generation approach of DeepSeek-Prover-V1, we propose RMaxTS, a
  variant of Monte-Carlo tree search that employs an intrinsic-reward-driven
  exploration strategy to generate diverse proof paths. DeepSeek-Prover-V1.5
  demonstrates significant improvements over DeepSeek-Prover-V1, achieving new
  state-of-the-art results on the test set of the high school level miniF2F
  benchmark (63.5%) and the undergraduate level ProofNet benchmark (25.3%).
paper_ai_summary: DeepSeek-Prover-V1.5 improves theorem proving by optimizing training and inference, utilizing reinforcement learning, and proposing RMaxTS for diverse proof paths, achieving state-of-the-art results on miniF2F and ProofNet benchmarks.
paper_ai_keywords: DeepSeek-Prover-V1.5 language model theorem proving Lean 4 DeepSeekMath-Base supervised fine-tuning RLPAF Monte-Carlo tree search RMaxTS intrinsic-reward-driven exploration miniF2F ProofNet