NILE: Internal Consistency Alignment in Large Language Models
As a crucial step to enhance LLMs alignment with human intentions,
Instruction Fine-Tuning (IFT) has a high demand on dataset quality. However,
existing IFT datasets often contain knowledge that is inconsistent with LLMs'
internal knowledge learned from the pre-training phase, which can greatly
affect the efficacy of IFT. To address this issue, we introduce NILE (iNternal
consIstency aLignmEnt) framework, aimed at optimizing IFT datasets to unlock
LLMs' capability further. NILE operates by eliciting target pre-trained LLM's
internal knowledge corresponding to instruction data. The internal knowledge is
leveraged to revise the answer in IFT datasets. Additionally, we propose a
novel Internal Consistency Filtering (ICF) method to filter training samples,
ensuring its high consistency with LLM's internal knowledge. Our experiments
demonstrate that NILE-aligned IFT datasets sharply boost LLM performance across
multiple LLM ability evaluation datasets, achieving up to 66.6% gain on
Arena-Hard and 68.5% on Alpaca-Eval V2. Further analysis confirms that each
component of the NILE}framework contributes to these substantial performance
improvements, and provides compelling evidence that dataset consistency with
pre-trained internal knowledge is pivotal for maximizing LLM potential.
676a20cbeabbef01bb5f825c
Minda Hu
claimed_verified
2024-12-30T19:35:43.586Z
65fd50dd78866973892e43eb
/avatars/f4a103b93a496c76537f27a8e4c11e6e.svg
Minda Hu
user
mindahu
676a20cbeabbef01bb5f825d
Qiyuan Zhang
claimed_verified
2024-12-30T19:35:45.781Z
62a42f22c683d02f5b63320c
/avatars/bc611abe9c4ef8d378123cb8ac9fdbf2.svg
Qiyuan Zhang
user
DonJoey
676a20cbeabbef01bb5f825e
Yufei Wang
676a20cbeabbef01bb5f825f
Bowei He
676a20cbeabbef01bb5f8260
Hongru Wang
676a20cbeabbef01bb5f8261
Jingyan Zhou
676a20cbeabbef01bb5f8262
Liangyou Li
676a20cbeabbef01bb5f8263
Yasheng Wang
676a20cbeabbef01bb5f8264
Chen Ma
676a20cbeabbef01bb5f8265
Irwin King
NILE: Internal Consistency Alignment in Large Language Models
As a crucial step to enhance LLMs alignment with human intentions,
Instruction Fine-Tuning (IFT) has a high demand on dataset quality. However,
existing IFT datasets often contain knowledge that is inconsistent with LLMs'
internal knowledge learned from the pre-training phase, which can greatly
affect the efficacy of IFT. To address this issue, we introduce NILE (iNternal
consIstency aLignmEnt) framework, aimed at optimizing IFT datasets to unlock
LLMs' capability further. NILE operates by eliciting target pre-trained LLM's
internal knowledge corresponding to instruction data. The internal knowledge is
leveraged to revise the answer in IFT datasets. Additionally, we propose a
novel Internal Consistency Filtering (ICF) method to filter training samples,
ensuring its high consistency with LLM's internal knowledge. Our experiments
demonstrate that NILE-aligned IFT datasets sharply boost LLM performance across
multiple LLM ability evaluation datasets, achieving up to 66.6% gain on
Arena-Hard and 68.5% on Alpaca-Eval V2. Further analysis confirms that each
component of the NILE}framework contributes to these substantial performance
improvements, and provides compelling evidence that dataset consistency with
pre-trained internal knowledge is pivotal for maximizing LLM potential.
The NILE framework optimizes Instruction Fine-Tuning datasets by aligning them with a pre-trained LLM's internal knowledge, significantly enhancing LLM performance across various evaluations.
Instruction Fine-Tuning IFT NILE internal knowledge Internal Consistency Filtering ICF Arena-Hard Alpaca-Eval V2 LLM Large Language Model