Connecting Federated ADMM to Bayes
Siddharth Swaroop Mohammad Emtiyaz Khan Finale Doshi-Velez
We provide new connections between two distinct federated learning approaches based on (i) ADMM and (ii) Variational Bayes (VB), and propose new variants by combining their complementary strengths. Specifically, we show that the dual variables in ADMM naturally emerge through the "site" parameters used in VB with isotropic Gaussian covariances. Using this, we derive two versions of ADMM from VB that use flexible covariances and functional regularisation, respectively. Through numerical experiments, we validate the improvements obtained in performance. The work shows connection between two fields that are believed to be fundamentally different and combines them to improve federated learning.
federated learning, bayesian, variational inference, admm
We provide connections between federated ADMM methods and Variational Bayes federated learning methods, providing ways to improve them