title: Iterative Graph Alignment
summary: By compressing diverse narratives, LLMs go beyond memorization, achieving
  intelligence by capturing generalizable causal relationships. However, they
  suffer from local 'representation gaps' due to insufficient training data
  diversity, limiting their real-world utility, especially in tasks requiring
  strict alignment to rules. Traditional alignment methods relying on heavy human
  annotations are inefficient and unscalable. Recent self-alignment techniques
  also fall short, as they often depend on self-selection based prompting and
  memorization-based learning. To address these issues, we introduce Iterative
  Graph Alignment (IGA), an annotation-free rule-based alignment algorithm. A
  teacher model (VLM) employs Iterative Graph Prompting (IGP) to create logical
  graphs and reference answers. The student model (LLM) identifies local
  knowledge gaps by attempting to align its responses with these references,
  collaborating with helper models to generate diverse answers. These aligned
  responses are then used for iterative supervised fine-tuning (SFT). Our
  evaluations across five rule-based scenarios demonstrate IGP's effectiveness,
  with a 73.12\% alignment improvement in Claude Sonnet 3.5, and
  Llama3-8B-Instruct achieving an 86.20\% improvement, outperforming Claude
  Sonnet 3.5 in rule-based alignment.
paper_authors._id: 66d11f2a092cc13d0f36e075
paper_authors.name: Fangyuan Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-30T07:18:05.215Z
paper_authors._id: 64d98ef7a4839890b25eb78b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64d98ef7a4839890b25eb78b/215-CSVLl81z6CAq0ECWU.jpeg
paper_authors.fullname: Fangyuan Yu
paper_authors.type: user
paper_authors.user: Ksgk-fy
paper_authors._id: 66d11f2a092cc13d0f36e076
paper_authors.name: Hardeep Singh Arora
paper_authors._id: 66d11f2a092cc13d0f36e077
paper_authors.name: Matt Johnson
paper_title: Iterative Graph Alignment
paper_summary: By compressing diverse narratives, LLMs go beyond memorization, achieving
  intelligence by capturing generalizable causal relationships. However, they
  suffer from local 'representation gaps' due to insufficient training data
  diversity, limiting their real-world utility, especially in tasks requiring
  strict alignment to rules. Traditional alignment methods relying on heavy human
  annotations are inefficient and unscalable. Recent self-alignment techniques
  also fall short, as they often depend on self-selection based prompting and
  memorization-based learning. To address these issues, we introduce Iterative
  Graph Alignment (IGA), an annotation-free rule-based alignment algorithm. A
  teacher model (VLM) employs Iterative Graph Prompting (IGP) to create logical
  graphs and reference answers. The student model (LLM) identifies local
  knowledge gaps by attempting to align its responses with these references,
  collaborating with helper models to generate diverse answers. These aligned
  responses are then used for iterative supervised fine-tuning (SFT). Our
  evaluations across five rule-based scenarios demonstrate IGP's effectiveness,
  with a 73.12\% alignment improvement in Claude Sonnet 3.5, and
  Llama3-8B-Instruct achieving an 86.20\% improvement, outperforming Claude
  Sonnet 3.5 in rule-based alignment.
paper_ai_keywords: LLMs (Large Language Models) VLM (Vision-Language Model) Iterative Graph Alignment (IGA) Iterative Graph Prompting (IGP) rule-based alignment annotation-free self-selection based prompting memorization-based learning logical graphs reference answers local knowledge gaps iterative supervised fine-tuning (SFT) rule-based scenarios