Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks
<|reference_start|>Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks: Federated Multi-Modal Learning (FMML) is an emerging field that integrates information from different modalities in federated learning to improve the learning performance. In this letter, we develop a parameter scheduling scheme to improve personalized performance and communication efficiency in personalized FMML, considering the non-independent and nonidentically distributed (non-IID) data along with the modality heterogeneity. Specifically, a learning-based approach is utilized to obtain the aggregation coefficients for parameters of different modalities on distinct devices. Based on the aggregation coefficients and channel state, a subset of parameters is scheduled to be uploaded to a server for each modality. Experimental results show that the proposed algorithm can effectively improve the personalized performance of FMML.<|reference_end|>
@article{yin2024aggregation,
    title={Aggregation Design for Personalized Federated Multi-Modal Learning over
  Wireless Networks},
    author={Benshun Yin, Zhiyong Chen, Meixia Tao},
    journal={arXiv preprint arXiv:2406.07915},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.07915},
    primaryClass={cs.IT eess.SP math.IT}
}