title: Low-Resource Machine Translation through the Lens of Personalized
    Federated Learning
summary: We present a new approach based on the Personalized Federated Learning
  algorithm MeritFed that can be applied to Natural Language Tasks with
  heterogeneous data. We evaluate it on the Low-Resource Machine Translation
  task, using the dataset from the Large-Scale Multilingual Machine Translation
  Shared Task (Small Track #2) and the subset of Sami languages from the
  multilingual benchmark for Finno-Ugric languages. In addition to its
  effectiveness, MeritFed is also highly interpretable, as it can be applied to
  track the impact of each language used for training. Our analysis reveals that
  target dataset size affects weight distribution across auxiliary languages,
  that unrelated languages do not interfere with the training, and auxiliary
  optimizer parameters have minimal impact. Our approach is easy to apply with a
  few lines of code, and we provide scripts for reproducing the experiments at
  https://github.com/VityaVitalich/MeritFed
paper_authors._id: 6679316cae4944f2cca7d1bf
paper_authors.name: Viktor Moskvoretskii
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T13:10:03.026Z
paper_authors._id: 63bbfd74141c7d395c471768
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
paper_authors.fullname: Viktor Moskvoretskii
paper_authors.type: user
paper_authors.user: VityaVitalich
paper_authors._id: 6679316cae4944f2cca7d1c0
paper_authors.name: Nazarii Tupitsa
paper_authors._id: 6679316cae4944f2cca7d1c1
paper_authors.name: Chris Biemann
paper_authors._id: 6679316cae4944f2cca7d1c2
paper_authors.name: Samuel Horváth
paper_authors._id: 6679316cae4944f2cca7d1c3
paper_authors.name: Eduard Gorbunov
paper_authors._id: 6679316cae4944f2cca7d1c4
paper_authors.name: Irina Nikishina
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T14:21:11.465Z
paper_authors._id: 6458a9ed6fa580137af4c710
paper_authors.avatarUrl: /avatars/8b9f99537adf30fee6aa7ae61ab7235b.svg
paper_authors.fullname: Irina Nikishina
paper_authors.type: user
paper_authors.user: lilaspourpre
paper_title: Low-Resource Machine Translation through the Lens of Personalized
    Federated Learning
paper_summary: We present a new approach based on the Personalized Federated Learning
  algorithm MeritFed that can be applied to Natural Language Tasks with
  heterogeneous data. We evaluate it on the Low-Resource Machine Translation
  task, using the dataset from the Large-Scale Multilingual Machine Translation
  Shared Task (Small Track #2) and the subset of Sami languages from the
  multilingual benchmark for Finno-Ugric languages. In addition to its
  effectiveness, MeritFed is also highly interpretable, as it can be applied to
  track the impact of each language used for training. Our analysis reveals that
  target dataset size affects weight distribution across auxiliary languages,
  that unrelated languages do not interfere with the training, and auxiliary
  optimizer parameters have minimal impact. Our approach is easy to apply with a
  few lines of code, and we provide scripts for reproducing the experiments at
  https://github.com/VityaVitalich/MeritFed
paper_ai_summary: MeritFed, a personalized federated learning algorithm, effectively handles low-resource machine translation with heterogeneous data and is interpretable in tracking language impacts.
paper_ai_keywords: Personalized Federated Learning MeritFed Federated Learning Low-Resource Machine Translation Large-Scale Multilingual Machine Translation auxiliary languages auxiliary optimizer parameters