title: O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple
    Distillation, Big Progress or Bitter Lesson?
summary: This paper presents a critical examination of current approaches to
  replicating OpenAI's O1 model capabilities, with particular focus on the
  widespread but often undisclosed use of knowledge distillation techniques.
  While our previous work explored the fundamental technical path to O1
  replication, this study reveals how simple distillation from O1's API, combined
  with supervised fine-tuning, can achieve superior performance on complex
  mathematical reasoning tasks. Through extensive experiments, we show that a
  base model fine-tuned on simply tens of thousands of samples O1-distilled
  long-thought chains outperforms O1-preview on the American Invitational
  Mathematics Examination (AIME) with minimal technical complexity. Moreover, our
  investigation extends beyond mathematical reasoning to explore the
  generalization capabilities of O1-distilled models across diverse tasks:
  hallucination, safety and open-domain QA. Notably, despite training only on
  mathematical problem-solving data, our models demonstrated strong
  generalization to open-ended QA tasks and became significantly less susceptible
  to sycophancy after fine-tuning. We deliberately make this finding public to
  promote transparency in AI research and to challenge the current trend of
  obscured technical claims in the field. Our work includes: (1) A detailed
  technical exposition of the distillation process and its effectiveness, (2) A
  comprehensive benchmark framework for evaluating and categorizing O1
  replication attempts based on their technical transparency and reproducibility,
  (3) A critical discussion of the limitations and potential risks of
  over-relying on distillation approaches, our analysis culminates in a crucial
  bitter lesson: while the pursuit of more capable AI systems is important, the
  development of researchers grounded in first-principles thinking is paramount.
paper_authors._id: 6745580279d6f3a9da792790
paper_authors.name: Zhen Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:43:14.316Z
paper_authors._id: 643581a4f3b08e267d990499
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643581a4f3b08e267d990499/KRhB-48W4IPuB0bX16Ahj.png
paper_authors.fullname: Zhen Huang
paper_authors.type: user
paper_authors.user: ZhenHuang
paper_authors._id: 6745580279d6f3a9da792791
paper_authors.name: Haoyang Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:43:08.617Z
paper_authors._id: 65011b18af1e4021439af578
paper_authors.avatarUrl: /avatars/ae1632c800ceeb43aa223e432f70bc6d.svg
paper_authors.fullname: Haoyang Zou
paper_authors.type: user
paper_authors.user: alanyoung058
paper_authors._id: 6745580279d6f3a9da792792
paper_authors.name: Xuefeng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:43:03.086Z
paper_authors._id: 64e3562342d8e2c1c69f64b3
paper_authors.avatarUrl: /avatars/1ccf22d60deec213bbe069d30811efbe.svg
paper_authors.fullname: xuefengli
paper_authors.type: user
paper_authors.user: xuefengli
paper_authors._id: 6745580279d6f3a9da792793
paper_authors.name: Yixiu Liu
paper_authors._id: 6745580279d6f3a9da792794
paper_authors.name: Yuxiang Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:13:57.942Z
paper_authors._id: 64dc2b8dff83b3386a5a897c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64dc2b8dff83b3386a5a897c/nxYnIXWmJRDuyw_EmqEAd.png
paper_authors.fullname: Yuxiang Zheng
paper_authors.type: user
paper_authors.user: catchiz
paper_authors._id: 6745580279d6f3a9da792795
paper_authors.name: Ethan Chern
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:42:37.230Z
paper_authors._id: 64bb5f9d8e051085bace4d1e
paper_authors.avatarUrl: /avatars/15ccbb78c6131dfe46b7a9d8e7d1a31f.svg
paper_authors.fullname: Ethan Chern
paper_authors.type: user
paper_authors.user: ethanchern
paper_authors._id: 6745580279d6f3a9da792796
paper_authors.name: Shijie Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:42:31.279Z
paper_authors._id: 65900d4ff5a209eeac08b463
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65900d4ff5a209eeac08b463/PJNNBRJIk1qR24oaRLTex.jpeg
paper_authors.fullname: shijie xia
paper_authors.type: user
paper_authors.user: seven-cat
paper_authors._id: 6745580279d6f3a9da792797
paper_authors.name: Yiwei Qin
paper_authors._id: 6745580279d6f3a9da792798
paper_authors.name: Weizhe Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:42:08.375Z
paper_authors._id: 6172aa78c8e66e2aa84c06b8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634904686602-noauth.png
paper_authors.fullname: Weizhe Yuan
paper_authors.type: user
paper_authors.user: weizhey
paper_authors._id: 6745580279d6f3a9da792799
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-26T22:42:01.710Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_title: O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple
    Distillation, Big Progress or Bitter Lesson?
paper_summary: This paper presents a critical examination of current approaches to
  replicating OpenAI's O1 model capabilities, with particular focus on the
  widespread but often undisclosed use of knowledge distillation techniques.
  While our previous work explored the fundamental technical path to O1
  replication, this study reveals how simple distillation from O1's API, combined
  with supervised fine-tuning, can achieve superior performance on complex
  mathematical reasoning tasks. Through extensive experiments, we show that a
  base model fine-tuned on simply tens of thousands of samples O1-distilled
  long-thought chains outperforms O1-preview on the American Invitational
  Mathematics Examination (AIME) with minimal technical complexity. Moreover, our
  investigation extends beyond mathematical reasoning to explore the
  generalization capabilities of O1-distilled models across diverse tasks:
  hallucination, safety and open-domain QA. Notably, despite training only on
  mathematical problem-solving data, our models demonstrated strong
  generalization to open-ended QA tasks and became significantly less susceptible
  to sycophancy after fine-tuning. We deliberately make this finding public to
  promote transparency in AI research and to challenge the current trend of
  obscured technical claims in the field. Our work includes: (1) A detailed
  technical exposition of the distillation process and its effectiveness, (2) A
  comprehensive benchmark framework for evaluating and categorizing O1
  replication attempts based on their technical transparency and reproducibility,
  (3) A critical discussion of the limitations and potential risks of
  over-relying on distillation approaches, our analysis culminates in a crucial
  bitter lesson: while the pursuit of more capable AI systems is important, the
  development of researchers grounded in first-principles thinking is paramount.
paper_ai_summary: Knowledge distillation from O1 model combined with fine-tuning achieves superior performance on complex mathematical reasoning and generalizes to open-ended QA tasks, promoting transparency in AI research.
paper_ai_keywords: knowledge distillation O1 model fine-tuning American Invitational Mathematics Examination (AIME) hallucination safety open-domain QA sycophancy