title: Towards Building the Federated GPT: Federated Instruction Tuning
summary: While ``instruction-tuned" generative large language models (LLMs) have
  demonstrated an impressive ability to generalize to new tasks, the training
  phases heavily rely on large amounts of diverse and high-quality instruction
  data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data,
  especially when it comes to human-written data, can pose significant challenges
  both in terms of cost and accessibility. Moreover, concerns related to privacy
  can further limit access to such data, making the process of obtaining it a
  complex and nuanced undertaking. Consequently, this hinders the generality of
  the tuned models and may restrict their effectiveness in certain contexts. To
  tackle this issue, our study introduces a new approach called Federated
  Instruction Tuning (FedIT), which leverages federated learning (FL) as the
  learning framework for the instruction tuning of LLMs. This marks the first
  exploration of FL-based instruction tuning for LLMs. This is especially
  important since text data is predominantly generated by end users. Therefore,
  it is imperative to design and adapt FL approaches to effectively leverage
  these users' diverse instructions stored on local devices, while preserving
  privacy and ensuring data security. In the current paper, by conducting widely
  used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous
  and diverse sets of instructions on the client's end with the proposed
  framework FedIT, we improved the performance of LLMs compared to centralized
  training with only limited local instructions. Further, in this paper, we
  developed a Github repository named Shepherd. This repository offers a
  foundational framework for exploring federated fine-tuning of LLMs using
  heterogeneous instructions across diverse categories.
paper_authors._id: 645aedca1a242120814700ab
paper_authors.name: Jianyi Zhang
paper_authors._id: 645aedca1a242120814700ac
paper_authors.name: Saeed Vahidian
paper_authors._id: 645aedca1a242120814700ad
paper_authors.name: Martin Kuo
paper_authors._id: 645aedca1a242120814700ae
paper_authors.name: Chunyuan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T14:27:42.468Z
paper_authors._id: 62aba526cae4462c0c6caa0f
paper_authors.avatarUrl: /avatars/430560ec2c2547f819225769ab432f30.svg
paper_authors.fullname: Chunyuan Li
paper_authors.type: user
paper_authors.user: Chunyuan24
paper_authors._id: 645aedca1a242120814700af
paper_authors.name: Ruiyi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-03T09:45:35.776Z
paper_authors._id: 64668f982da1abc242355cbb
paper_authors.avatarUrl: /avatars/c9a248b8d70b1a8f7b78265c98690570.svg
paper_authors.fullname: Ruiyi Zhang
paper_authors.type: user
paper_authors.user: zhangry868
paper_authors._id: 645aedca1a242120814700b0
paper_authors.name: Guoyin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T11:18:17.737Z
paper_authors._id: 6490d4ba1afdee3acd1147f6
paper_authors.avatarUrl: /avatars/ae13c7b21fe9ced7541dcd664d1b94ed.svg
paper_authors.fullname: Guoyin Wang
paper_authors.type: user
paper_authors.user: guoyinwang
paper_authors._id: 645aedca1a242120814700b1
paper_authors.name: Yiran Chen
paper_title: Towards Building the Federated GPT: Federated Instruction Tuning
paper_summary: While ``instruction-tuned" generative large language models (LLMs) have
  demonstrated an impressive ability to generalize to new tasks, the training
  phases heavily rely on large amounts of diverse and high-quality instruction
  data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data,
  especially when it comes to human-written data, can pose significant challenges
  both in terms of cost and accessibility. Moreover, concerns related to privacy
  can further limit access to such data, making the process of obtaining it a
  complex and nuanced undertaking. Consequently, this hinders the generality of
  the tuned models and may restrict their effectiveness in certain contexts. To
  tackle this issue, our study introduces a new approach called Federated
  Instruction Tuning (FedIT), which leverages federated learning (FL) as the
  learning framework for the instruction tuning of LLMs. This marks the first
  exploration of FL-based instruction tuning for LLMs. This is especially
  important since text data is predominantly generated by end users. Therefore,
  it is imperative to design and adapt FL approaches to effectively leverage
  these users' diverse instructions stored on local devices, while preserving
  privacy and ensuring data security. In the current paper, by conducting widely
  used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous
  and diverse sets of instructions on the client's end with the proposed
  framework FedIT, we improved the performance of LLMs compared to centralized
  training with only limited local instructions. Further, in this paper, we
  developed a Github repository named Shepherd. This repository offers a
  foundational framework for exploring federated fine-tuning of LLMs using
  heterogeneous instructions across diverse categories.
paper_ai_summary: Federated Instruction Tuning (FedIT) leverages federated learning to improve generative large language models with diverse local instructions while maintaining privacy.
paper_ai_keywords: generative large language models (LLMs) instruction-tuned federated learning (FL) Federated Instruction Tuning (FedIT) GPT-4 auto-evaluation heterogeneous client's end federated fine-tuning