title: Exploring Format Consistency for Instruction Tuning
summary: Instruction tuning has emerged as a promising approach to enhancing large
  language models in following human instructions. It is shown that increasing
  the diversity and number of instructions in the training data can consistently
  enhance generalization performance, which facilitates a recent endeavor to
  collect various instructions and integrate existing instruction tuning datasets
  into larger collections. However, different users have their unique ways of
  expressing instructions, and there often exist variations across different
  datasets in the instruction styles and formats, i.e., format inconsistency. In
  this work, we study how format inconsistency may impact the performance of
  instruction tuning. We propose a framework called "Unified Instruction Tuning"
  (UIT), which calls OpenAI APIs for automatic format transfer among different
  instruction tuning datasets. We show that UIT successfully improves the
  generalization performance on unseen instructions, which highlights the
  importance of format consistency for instruction tuning. To make the UIT
  framework more practical, we further propose a novel perplexity-based denoising
  method to reduce the noise of automatic format transfer. We also train a
  smaller offline model that achieves comparable format transfer capability than
  OpenAI APIs to reduce costs in practice.
paper_authors._id: 64c749e02d07296c7e677157
paper_authors.name: Shihao Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-31T07:14:16.374Z
paper_authors._id: 6241d6b4bdf59d13c0a8dd86
paper_authors.avatarUrl: /avatars/f729f0bdcfbd971babfa760d8de8403c.svg
paper_authors.fullname: Shihao Liang
paper_authors.type: user
paper_authors.user: pooruss
paper_authors._id: 64c749e02d07296c7e677158
paper_authors.name: Kunlun Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-31T07:14:46.475Z
paper_authors._id: 64c090a9f613170e7be93d2f
paper_authors.avatarUrl: /avatars/ccbdf444e1f2386d2281e8e42059ebb0.svg
paper_authors.fullname: KunlunZhu
paper_authors.type: user
paper_authors.user: KunlunZhu
paper_authors._id: 64c749e02d07296c7e677159
paper_authors.name: Runchu Tian
paper_authors._id: 64c749e02d07296c7e67715a
paper_authors.name: Yujia Qin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-31T07:15:30.400Z
paper_authors._id: 643f37cce9d063936912048b
paper_authors.avatarUrl: /avatars/25822ea5676a79b2e1ddf08d5fc2226c.svg
paper_authors.fullname: Yujia Qin
paper_authors.type: user
paper_authors.user: YujiaHi
paper_authors._id: 64c749e02d07296c7e67715b
paper_authors.name: Huadong Wang
paper_authors._id: 64c749e02d07296c7e67715c
paper_authors.name: Xin Cong
paper_authors._id: 64c749e02d07296c7e67715d
paper_authors.name: Zhiyuan Liu
paper_authors._id: 64c749e02d07296c7e67715e
paper_authors.name: Xiaojiang Liu
paper_authors._id: 64c749e02d07296c7e67715f
paper_authors.name: Maosong Sun
paper_title: Exploring Format Consistency for Instruction Tuning
paper_summary: Instruction tuning has emerged as a promising approach to enhancing large
  language models in following human instructions. It is shown that increasing
  the diversity and number of instructions in the training data can consistently
  enhance generalization performance, which facilitates a recent endeavor to
  collect various instructions and integrate existing instruction tuning datasets
  into larger collections. However, different users have their unique ways of
  expressing instructions, and there often exist variations across different
  datasets in the instruction styles and formats, i.e., format inconsistency. In
  this work, we study how format inconsistency may impact the performance of
  instruction tuning. We propose a framework called "Unified Instruction Tuning"
  (UIT), which calls OpenAI APIs for automatic format transfer among different
  instruction tuning datasets. We show that UIT successfully improves the
  generalization performance on unseen instructions, which highlights the
  importance of format consistency for instruction tuning. To make the UIT
  framework more practical, we further propose a novel perplexity-based denoising
  method to reduce the noise of automatic format transfer. We also train a
  smaller offline model that achieves comparable format transfer capability than
  OpenAI APIs to reduce costs in practice.
paper_ai_keywords: instruction tuning generalization performance instruction styles format inconsistency Unified Instruction Tuning (UIT) OpenAI APIs automatic format transfer perplexity-based denoising method