title: I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative
    Self-Enhancement Paradigm
summary: Large Language Models (LLMs) have achieved significant advancements, however,
  the common learning paradigm treats LLMs as passive information repositories,
  neglecting their potential for active learning and alignment. Some approaches
  train LLMs using their own generated synthetic data, exploring the possibility
  of active alignment. However, there is still a huge gap between these one-time
  alignment methods and the continuous automatic alignment of humans. In this
  paper, we introduce I-SHEEP, an Iterative
  Self-EnHancEmEnt Paradigm.This
  human-like paradigm enables LLMs to continuously self-align from
  scratch with nothing. Compared to the one-time alignment method Dromedary
  sun2023principledriven, which refers to the first iteration in this
  paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama
  models. I-SHEEP achieves a maximum relative improvement of 78.2\% in the Alpaca
  Eval, 24.0\% in the MT Bench, and an absolute increase of 8.88\% in the IFEval
  accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally,
  I-SHEEP surpasses the base model in various standard benchmark generation
  tasks, achieving an average improvement of 24.77\% in code generation tasks,
  12.04\% in TrivialQA, and 20.29\% in SQuAD. We also provide new insights based
  on the experiment results. Our codes, datasets, and models are available at
  https://anonymous.4open.science/r/I-SHEEP.
paper_authors._id: 66bedfa3ac74db25de04bd12
paper_authors.name: Yiming Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-16T09:20:26.552Z
paper_authors._id: 6555e8d8a0c34cd61a6b9ce3
paper_authors.avatarUrl: /avatars/71dc562cef4bd42f6b762f036357c800.svg
paper_authors.fullname: yimingliang
paper_authors.type: user
paper_authors.user: yimingliang
paper_authors._id: 66bedfa3ac74db25de04bd13
paper_authors.name: Ge Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-21T07:16:48.215Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 66bedfa3ac74db25de04bd14
paper_authors.name: Xingwei Qu
paper_authors._id: 66bedfa3ac74db25de04bd15
paper_authors.name: Tianyu Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:53:45.446Z
paper_authors._id: 64ab99dcb76bfd863eba64c1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
paper_authors.fullname: TY.Zheng
paper_authors.type: user
paper_authors.user: aaabiao
paper_authors._id: 66bedfa3ac74db25de04bd16
paper_authors.name: Jiawei Guo
paper_authors._id: 66bedfa3ac74db25de04bd17
paper_authors.name: Xinrun Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-03T14:24:53.513Z
paper_authors._id: 654907a4a1faff97850c4eff
paper_authors.avatarUrl: /avatars/458c90151614bc7f116943b6e67d6b8a.svg
paper_authors.fullname: du
paper_authors.type: user
paper_authors.user: dododododo
paper_authors._id: 66bedfa3ac74db25de04bd18
paper_authors.name: Zhenzhu Yang
paper_authors._id: 66bedfa3ac74db25de04bd19
paper_authors.name: Jiaheng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-05T23:03:36.525Z
paper_authors._id: 65377c30e48353201e6fdda0
paper_authors.avatarUrl: /avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
paper_authors.fullname: Jiaheng Liu
paper_authors.type: user
paper_authors.user: CheeryLJH
paper_authors._id: 66bedfa3ac74db25de04bd1a
paper_authors.name: Chenghua Lin
paper_authors._id: 66bedfa3ac74db25de04bd1b
paper_authors.name: Lei Ma
paper_authors._id: 66bedfa3ac74db25de04bd1c
paper_authors.name: Wenhao Huang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-08-16T05:12:04.738Z
paper_authors._id: 635c855dbb96f709230e8acd
paper_authors.avatarUrl: /avatars/8b580863425f6d5cfa9ce833eb4c6d34.svg
paper_authors.fullname: Rubio Huang
paper_authors.type: user
paper_authors.user: Rubio41
paper_authors._id: 66bedfa3ac74db25de04bd1d
paper_authors.name: Jiajun Zhang
paper_title: I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative
    Self-Enhancement Paradigm
paper_summary: Large Language Models (LLMs) have achieved significant advancements, however,
  the common learning paradigm treats LLMs as passive information repositories,
  neglecting their potential for active learning and alignment. Some approaches
  train LLMs using their own generated synthetic data, exploring the possibility
  of active alignment. However, there is still a huge gap between these one-time
  alignment methods and the continuous automatic alignment of humans. In this
  paper, we introduce I-SHEEP, an Iterative
  Self-EnHancEmEnt Paradigm.This
  human-like paradigm enables LLMs to continuously self-align from
  scratch with nothing. Compared to the one-time alignment method Dromedary
  sun2023principledriven, which refers to the first iteration in this
  paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama
  models. I-SHEEP achieves a maximum relative improvement of 78.2\% in the Alpaca
  Eval, 24.0\% in the MT Bench, and an absolute increase of 8.88\% in the IFEval
  accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally,
  I-SHEEP surpasses the base model in various standard benchmark generation
  tasks, achieving an average improvement of 24.77\% in code generation tasks,
  12.04\% in TrivialQA, and 20.29\% in SQuAD. We also provide new insights based
  on the experiment results. Our codes, datasets, and models are available at
  https://anonymous.4open.science/r/I-SHEEP.
paper_ai_keywords: I-SHEEP Iterative Self-Enhancement Paradigm active alignment synthetic data human-like paradigm continuous self-align