title: Exploring Federated Pruning for Large Language Models
summary: LLM pruning has emerged as a promising technology for compressing LLMs,
  enabling their deployment on resource-limited devices. However, current
  methodologies typically require access to public calibration samples, which can
  be challenging to obtain in privacy-sensitive domains. To address this issue,
  we introduce FedPrLLM, a comprehensive federated pruning framework designed for
  the privacy-preserving compression of LLMs. In FedPrLLM, each client only needs
  to calculate a pruning mask matrix based on its local calibration data and
  share it with the server to prune the global model. This approach allows for
  collaborative pruning of the global model with the knowledge of each client
  while maintaining local data privacy. Additionally, we conduct extensive
  experiments to explore various possibilities within the FedPrLLM framework,
  including different comparison groups, pruning strategies, and the decision to
  scale weights. Our extensive evaluation reveals that one-shot pruning with
  layer comparison and no weight scaling is the optimal choice within the
  FedPrLLM framework. We hope our work will help guide future efforts in pruning
  LLMs in privacy-sensitive fields. Our code is available at
  https://github.com/Pengxin-Guo/FedPrLLM.
paper_authors._id: 682d6b06ec3b65b35772c0af
paper_authors.name: Pengxin Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:40:11.271Z
paper_authors._id: 668f440894dfc0ed1a7006ed
paper_authors.avatarUrl: /avatars/fa0d328300b03bcbbf9b3a7532f28458.svg
paper_authors.fullname: Pengxin Guo
paper_authors.type: user
paper_authors.user: gpx333
paper_authors._id: 682d6b06ec3b65b35772c0b0
paper_authors.name: Yinong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:12:52.463Z
paper_authors._id: 641b065a1911d3be6742ef04
paper_authors.avatarUrl: /avatars/bceb4ad2a278b6e2a40af0b89c4fb48e.svg
paper_authors.fullname: Yinong Wang
paper_authors.type: user
paper_authors.user: jcccy
paper_authors._id: 682d6b06ec3b65b35772c0b1
paper_authors.name: Wei Li
paper_authors._id: 682d6b06ec3b65b35772c0b2
paper_authors.name: Mengting Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:13:13.691Z
paper_authors._id: 678f8d4b1a5e393c1c285165
paper_authors.avatarUrl: /avatars/533eb74376915eb63c03dc9b4df421f6.svg
paper_authors.fullname: MENGTINGLIU
paper_authors.type: user
paper_authors.user: MENGTINGLIU
paper_authors._id: 682d6b06ec3b65b35772c0b3
paper_authors.name: Ming Li
paper_authors._id: 682d6b06ec3b65b35772c0b4
paper_authors.name: Jinkai Zheng
paper_authors._id: 682d6b06ec3b65b35772c0b5
paper_authors.name: Liangqiong Qu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:12:59.427Z
paper_authors._id: 663058bc2653ec94f4a6235f
paper_authors.avatarUrl: /avatars/f55b8c3c8100d6b6d65ba61abc4fb014.svg
paper_authors.fullname: Liangqiong Qu
paper_authors.type: user
paper_authors.user: Liangqiong-QU
paper_title: Exploring Federated Pruning for Large Language Models
paper_summary: LLM pruning has emerged as a promising technology for compressing LLMs,
  enabling their deployment on resource-limited devices. However, current
  methodologies typically require access to public calibration samples, which can
  be challenging to obtain in privacy-sensitive domains. To address this issue,
  we introduce FedPrLLM, a comprehensive federated pruning framework designed for
  the privacy-preserving compression of LLMs. In FedPrLLM, each client only needs
  to calculate a pruning mask matrix based on its local calibration data and
  share it with the server to prune the global model. This approach allows for
  collaborative pruning of the global model with the knowledge of each client
  while maintaining local data privacy. Additionally, we conduct extensive
  experiments to explore various possibilities within the FedPrLLM framework,
  including different comparison groups, pruning strategies, and the decision to
  scale weights. Our extensive evaluation reveals that one-shot pruning with
  layer comparison and no weight scaling is the optimal choice within the
  FedPrLLM framework. We hope our work will help guide future efforts in pruning
  LLMs in privacy-sensitive fields. Our code is available at
  https://github.com/Pengxin-Guo/FedPrLLM.
paper_ai_summary: FedPrLLM is a federated pruning framework that enables privacy-preserving compression of large language models (LLMs) using local client data.
paper_ai_keywords: FedPrLLM federated pruning privacy-preserving compression LLMs calibration data pruning mask matrix layer comparison weight scaling