Exploring Format Consistency for Instruction Tuning
Instruction tuning has emerged as a promising approach to enhancing large
language models in following human instructions. It is shown that increasing
the diversity and number of instructions in the training data can consistently
enhance generalization performance, which facilitates a recent endeavor to
collect various instructions and integrate existing instruction tuning datasets
into larger collections. However, different users have their unique ways of
expressing instructions, and there often exist variations across different
datasets in the instruction styles and formats, i.e., format inconsistency. In
this work, we study how format inconsistency may impact the performance of
instruction tuning. We propose a framework called "Unified Instruction Tuning"
(UIT), which calls OpenAI APIs for automatic format transfer among different
instruction tuning datasets. We show that UIT successfully improves the
generalization performance on unseen instructions, which highlights the
importance of format consistency for instruction tuning. To make the UIT
framework more practical, we further propose a novel perplexity-based denoising
method to reduce the noise of automatic format transfer. We also train a
smaller offline model that achieves comparable format transfer capability than
OpenAI APIs to reduce costs in practice.
64c749e02d07296c7e677157
Shihao Liang
admin_assigned
2023-07-31T07:14:16.374Z
6241d6b4bdf59d13c0a8dd86
/avatars/f729f0bdcfbd971babfa760d8de8403c.svg
Shihao Liang
user
pooruss
64c749e02d07296c7e677158
Kunlun Zhu
admin_assigned
2023-07-31T07:14:46.475Z
64c090a9f613170e7be93d2f
/avatars/ccbdf444e1f2386d2281e8e42059ebb0.svg
KunlunZhu
user
KunlunZhu
64c749e02d07296c7e677159
Runchu Tian
64c749e02d07296c7e67715a
Yujia Qin
admin_assigned
2023-07-31T07:15:30.400Z
643f37cce9d063936912048b
/avatars/25822ea5676a79b2e1ddf08d5fc2226c.svg
Yujia Qin
user
YujiaHi
64c749e02d07296c7e67715b
Huadong Wang
64c749e02d07296c7e67715c
Xin Cong
64c749e02d07296c7e67715d
Zhiyuan Liu
claimed_verified
2025-06-02T08:02:03.463Z
6472f1f7485a7c8e1bcf507b
/avatars/79dca137583875191ea883b193b630b7.svg
Zhiyuan Liu
user
zibuyu9
64c749e02d07296c7e67715e
Xiaojiang Liu
64c749e02d07296c7e67715f
Maosong Sun
Exploring Format Consistency for Instruction Tuning
Instruction tuning has emerged as a promising approach to enhancing large
language models in following human instructions. It is shown that increasing
the diversity and number of instructions in the training data can consistently
enhance generalization performance, which facilitates a recent endeavor to
collect various instructions and integrate existing instruction tuning datasets
into larger collections. However, different users have their unique ways of
expressing instructions, and there often exist variations across different
datasets in the instruction styles and formats, i.e., format inconsistency. In
this work, we study how format inconsistency may impact the performance of
instruction tuning. We propose a framework called "Unified Instruction Tuning"
(UIT), which calls OpenAI APIs for automatic format transfer among different
instruction tuning datasets. We show that UIT successfully improves the
generalization performance on unseen instructions, which highlights the
importance of format consistency for instruction tuning. To make the UIT
framework more practical, we further propose a novel perplexity-based denoising
method to reduce the noise of automatic format transfer. We also train a
smaller offline model that achieves comparable format transfer capability than
OpenAI APIs to reduce costs in practice.
Unified Instruction Tuning improves instruction tuning performance by addressing format inconsistencies across datasets using automatic format transfer and denoising techniques.
instruction tuning large language models generalization performance instruction styles format inconsistency Unified Instruction Tuning UIT OpenAI APIs perplexity-based denoising offline model