Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding
<|reference_start|>Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding: The integration of Federated Learning (FL) and Self-supervised Learning (SSL) offers a unique and synergetic combination to exploit the audio data for general-purpose audio understanding, without compromising user data privacy. However, rare efforts have been made to investigate the SSL models in the FL regime for general-purpose audio understanding, especially when the training data is generated by large-scale heterogeneous audio sources. In this paper, we evaluate the performance of feature-matching and predictive audio-SSL techniques when integrated into large-scale FL settings simulated with non-independently identically distributed (non-iid) data. We propose a novel Federated SSL (F-SSL) framework, dubbed FASSL, that enables learning intermediate feature representations from large-scale decentralized heterogeneous clients, holding unlabelled audio data. Our study has found that audio F-SSL approaches perform on par with the centralized audio-SSL approaches on the audio-retrieval task. Extensive experiments demonstrate the effectiveness and significance of FASSL as it assists in obtaining the optimal global model for state-of-the-art FL aggregation methods.<|reference_end|>
@article{rehman2024exploring,
    title={Exploring Federated Self-Supervised Learning for General Purpose Audio
  Understanding},
    author={Yasar Abbas Ur Rehman, Kin Wai Lau, Yuyang Xie, Lan Ma, Jiajun Shen},
    journal={arXiv preprint arXiv:2402.02889},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.02889},
    primaryClass={cs.SD cs.CV cs.LG eess.AS}
}