FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA
  Subparameter Updates
Federated Learning (FL) is a widely used framework for training models in a
decentralized manner, ensuring that the central server does not have direct
access to data from local clients. However, this approach may still fail to
fully preserve data privacy, as models from local clients are exposed to the
central server during the aggregation process. This issue becomes even more
critical when training vision-language models (VLMs) with FL, as VLMs can
easily memorize training data instances, making them vulnerable to membership
inference attacks (MIAs). To address this challenge, we propose the FedRand
framework, which avoids disclosing the full set of client parameters. In this
framework, each client randomly selects subparameters of Low-Rank Adaptation
(LoRA) from the server and keeps the remaining counterparts of the LoRA weights
as private parameters. After training both parameters on the client's private
dataset, only the non-private client parameters are sent back to the server for
aggregation. This approach mitigates the risk of exposing client-side VLM
parameters, thereby enhancing data privacy. We empirically validate that
FedRand improves robustness against MIAs compared to relevant baselines while
achieving accuracy comparable to methods that communicate full LoRA parameters
across several benchmark datasets.
67cfa6fcd77496ce0c154bdc
Sangwoo Park
claimed_verified
2025-03-11T11:43:53.415Z
638716c14e00d7fc0902fef4
/avatars/5fa8152f8c0e4e600d1a64802c3e0103.svg
Sangwoo Park
user
Sangsang
67cfa6fcd77496ce0c154bdd
Seanie Lee
admin_assigned
2025-03-11T11:02:25.920Z
64ad5f59b7e4b2c1ce47eb43
/avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg
Seanie Lee
user
Seanie-lee
67cfa6fcd77496ce0c154bde
Byungjoo Kim
admin_assigned
2025-03-11T11:02:08.406Z
65f2ab29d83aee7a6492b7b7
/avatars/d0ff817329ee9a534a936dad41a1943e.svg
Byungjoo Kim
user
matbambbang
67cfa6fcd77496ce0c154bdf
Sung Ju Hwang
FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA
  Subparameter Updates
Federated Learning (FL) is a widely used framework for training models in a
decentralized manner, ensuring that the central server does not have direct
access to data from local clients. However, this approach may still fail to
fully preserve data privacy, as models from local clients are exposed to the
central server during the aggregation process. This issue becomes even more
critical when training vision-language models (VLMs) with FL, as VLMs can
easily memorize training data instances, making them vulnerable to membership
inference attacks (MIAs). To address this challenge, we propose the FedRand
framework, which avoids disclosing the full set of client parameters. In this
framework, each client randomly selects subparameters of Low-Rank Adaptation
(LoRA) from the server and keeps the remaining counterparts of the LoRA weights
as private parameters. After training both parameters on the client's private
dataset, only the non-private client parameters are sent back to the server for
aggregation. This approach mitigates the risk of exposing client-side VLM
parameters, thereby enhancing data privacy. We empirically validate that
FedRand improves robustness against MIAs compared to relevant baselines while
achieving accuracy comparable to methods that communicate full LoRA parameters
across several benchmark datasets.
FedRand framework enhances data privacy in federated learning by keeping a subset of LoRA parameters private, reducing the risk of membership inference attacks while maintaining model accuracy.
Federated Learning vision-language models membership inference attacks Low-Rank Adaptation LoRA FedRand