title: NILE: Internal Consistency Alignment in Large Language Models
summary: As a crucial step to enhance LLMs alignment with human intentions,
  Instruction Fine-Tuning (IFT) has a high demand on dataset quality. However,
  existing IFT datasets often contain knowledge that is inconsistent with LLMs'
  internal knowledge learned from the pre-training phase, which can greatly
  affect the efficacy of IFT. To address this issue, we introduce NILE (iNternal
  consIstency aLignmEnt) framework, aimed at optimizing IFT datasets to unlock
  LLMs' capability further. NILE operates by eliciting target pre-trained LLM's
  internal knowledge corresponding to instruction data. The internal knowledge is
  leveraged to revise the answer in IFT datasets. Additionally, we propose a
  novel Internal Consistency Filtering (ICF) method to filter training samples,
  ensuring its high consistency with LLM's internal knowledge. Our experiments
  demonstrate that NILE-aligned IFT datasets sharply boost LLM performance across
  multiple LLM ability evaluation datasets, achieving up to 66.6% gain on
  Arena-Hard and 68.5% on Alpaca-Eval V2. Further analysis confirms that each
  component of the NILE}framework contributes to these substantial performance
  improvements, and provides compelling evidence that dataset consistency with
  pre-trained internal knowledge is pivotal for maximizing LLM potential.
paper_authors._id: 676a20cbeabbef01bb5f825c
paper_authors.name: Minda Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:35:43.586Z
paper_authors._id: 65fd50dd78866973892e43eb
paper_authors.avatarUrl: /avatars/f4a103b93a496c76537f27a8e4c11e6e.svg
paper_authors.fullname: Minda Hu
paper_authors.type: user
paper_authors.user: mindahu
paper_authors._id: 676a20cbeabbef01bb5f825d
paper_authors.name: Qiyuan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:35:45.781Z
paper_authors._id: 62a42f22c683d02f5b63320c
paper_authors.avatarUrl: /avatars/bc611abe9c4ef8d378123cb8ac9fdbf2.svg
paper_authors.fullname: Qiyuan Zhang
paper_authors.type: user
paper_authors.user: DonJoey
paper_authors._id: 676a20cbeabbef01bb5f825e
paper_authors.name: Yufei Wang
paper_authors._id: 676a20cbeabbef01bb5f825f
paper_authors.name: Bowei He
paper_authors._id: 676a20cbeabbef01bb5f8260
paper_authors.name: Hongru Wang
paper_authors._id: 676a20cbeabbef01bb5f8261
paper_authors.name: Jingyan Zhou
paper_authors._id: 676a20cbeabbef01bb5f8262
paper_authors.name: Liangyou Li
paper_authors._id: 676a20cbeabbef01bb5f8263
paper_authors.name: Yasheng Wang
paper_authors._id: 676a20cbeabbef01bb5f8264
paper_authors.name: Chen Ma
paper_authors._id: 676a20cbeabbef01bb5f8265
paper_authors.name: Irwin King
paper_title: NILE: Internal Consistency Alignment in Large Language Models
paper_summary: As a crucial step to enhance LLMs alignment with human intentions,
  Instruction Fine-Tuning (IFT) has a high demand on dataset quality. However,
  existing IFT datasets often contain knowledge that is inconsistent with LLMs'
  internal knowledge learned from the pre-training phase, which can greatly
  affect the efficacy of IFT. To address this issue, we introduce NILE (iNternal
  consIstency aLignmEnt) framework, aimed at optimizing IFT datasets to unlock
  LLMs' capability further. NILE operates by eliciting target pre-trained LLM's
  internal knowledge corresponding to instruction data. The internal knowledge is
  leveraged to revise the answer in IFT datasets. Additionally, we propose a
  novel Internal Consistency Filtering (ICF) method to filter training samples,
  ensuring its high consistency with LLM's internal knowledge. Our experiments
  demonstrate that NILE-aligned IFT datasets sharply boost LLM performance across
  multiple LLM ability evaluation datasets, achieving up to 66.6% gain on
  Arena-Hard and 68.5% on Alpaca-Eval V2. Further analysis confirms that each
  component of the NILE}framework contributes to these substantial performance
  improvements, and provides compelling evidence that dataset consistency with
  pre-trained internal knowledge is pivotal for maximizing LLM potential.
paper_ai_keywords: Instruction Fine-Tuning (IFT) NILE (iNternal consIstency aLignmEnt) pre-trained LLM's internal knowledge IFT datasets Internal Consistency Filtering (ICF) LLM performance Arena-Hard Alpaca-Eval V2