title: FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA
    Subparameter Updates
summary: Federated Learning (FL) is a widely used framework for training models in a
  decentralized manner, ensuring that the central server does not have direct
  access to data from local clients. However, this approach may still fail to
  fully preserve data privacy, as models from local clients are exposed to the
  central server during the aggregation process. This issue becomes even more
  critical when training vision-language models (VLMs) with FL, as VLMs can
  easily memorize training data instances, making them vulnerable to membership
  inference attacks (MIAs). To address this challenge, we propose the FedRand
  framework, which avoids disclosing the full set of client parameters. In this
  framework, each client randomly selects subparameters of Low-Rank Adaptation
  (LoRA) from the server and keeps the remaining counterparts of the LoRA weights
  as private parameters. After training both parameters on the client's private
  dataset, only the non-private client parameters are sent back to the server for
  aggregation. This approach mitigates the risk of exposing client-side VLM
  parameters, thereby enhancing data privacy. We empirically validate that
  FedRand improves robustness against MIAs compared to relevant baselines while
  achieving accuracy comparable to methods that communicate full LoRA parameters
  across several benchmark datasets.
paper_authors._id: 67cfa6fcd77496ce0c154bdc
paper_authors.name: Sangwoo Park
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T11:43:53.415Z
paper_authors._id: 638716c14e00d7fc0902fef4
paper_authors.avatarUrl: /avatars/5fa8152f8c0e4e600d1a64802c3e0103.svg
paper_authors.fullname: Sangwoo Park
paper_authors.type: user
paper_authors.user: Sangsang
paper_authors._id: 67cfa6fcd77496ce0c154bdd
paper_authors.name: Seanie Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T11:02:25.920Z
paper_authors._id: 64ad5f59b7e4b2c1ce47eb43
paper_authors.avatarUrl: /avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg
paper_authors.fullname: Seanie Lee
paper_authors.type: user
paper_authors.user: Seanie-lee
paper_authors._id: 67cfa6fcd77496ce0c154bde
paper_authors.name: Byungjoo Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T11:02:08.406Z
paper_authors._id: 65f2ab29d83aee7a6492b7b7
paper_authors.avatarUrl: /avatars/d0ff817329ee9a534a936dad41a1943e.svg
paper_authors.fullname: Byungjoo Kim
paper_authors.type: user
paper_authors.user: matbambbang
paper_authors._id: 67cfa6fcd77496ce0c154bdf
paper_authors.name: Sung Ju Hwang
paper_title: FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA
    Subparameter Updates
paper_summary: Federated Learning (FL) is a widely used framework for training models in a
  decentralized manner, ensuring that the central server does not have direct
  access to data from local clients. However, this approach may still fail to
  fully preserve data privacy, as models from local clients are exposed to the
  central server during the aggregation process. This issue becomes even more
  critical when training vision-language models (VLMs) with FL, as VLMs can
  easily memorize training data instances, making them vulnerable to membership
  inference attacks (MIAs). To address this challenge, we propose the FedRand
  framework, which avoids disclosing the full set of client parameters. In this
  framework, each client randomly selects subparameters of Low-Rank Adaptation
  (LoRA) from the server and keeps the remaining counterparts of the LoRA weights
  as private parameters. After training both parameters on the client's private
  dataset, only the non-private client parameters are sent back to the server for
  aggregation. This approach mitigates the risk of exposing client-side VLM
  parameters, thereby enhancing data privacy. We empirically validate that
  FedRand improves robustness against MIAs compared to relevant baselines while
  achieving accuracy comparable to methods that communicate full LoRA parameters
  across several benchmark datasets.
paper_ai_summary: FedRand framework enhances data privacy in federated learning by keeping a subset of LoRA parameters private, reducing the risk of membership inference attacks while maintaining model accuracy.
paper_ai_keywords: Federated Learning vision-language models membership inference attacks Low-Rank Adaptation LoRA FedRand