title: Expanding FLORES+ Benchmark for more Low-Resource Settings:
    Portuguese-Emakhuwa Machine Translation Evaluation
summary: As part of the Open Language Data Initiative shared tasks, we have expanded
  the FLORES+ evaluation set to include Emakhuwa, a low-resource language widely
  spoken in Mozambique. We translated the dev and devtest sets from Portuguese
  into Emakhuwa, and we detail the translation process and quality assurance
  measures used. Our methodology involved various quality checks, including
  post-editing and adequacy assessments. The resulting datasets consist of
  multiple reference sentences for each source. We present baseline results from
  training a Neural Machine Translation system and fine-tuning existing
  multilingual translation models. Our findings suggest that spelling
  inconsistencies remain a challenge in Emakhuwa. Additionally, the baseline
  models underperformed on this evaluation set, underscoring the necessity for
  further research to enhance machine translation quality for Emakhuwa. The data
  is publicly available at https://huggingface.co/datasets/LIACC/Emakhuwa-FLORES.
paper_authors._id: 66c7004f60ef51a02f30164f
paper_authors.name: Felermino D. M. Antonio Ali
paper_authors._id: 66c7004f60ef51a02f301650
paper_authors.name: Henrique Lopes Cardoso
paper_authors._id: 66c7004f60ef51a02f301651
paper_authors.name: Rui Sousa-Silva
paper_title: Expanding FLORES+ Benchmark for more Low-Resource Settings:
    Portuguese-Emakhuwa Machine Translation Evaluation
paper_summary: As part of the Open Language Data Initiative shared tasks, we have expanded
  the FLORES+ evaluation set to include Emakhuwa, a low-resource language widely
  spoken in Mozambique. We translated the dev and devtest sets from Portuguese
  into Emakhuwa, and we detail the translation process and quality assurance
  measures used. Our methodology involved various quality checks, including
  post-editing and adequacy assessments. The resulting datasets consist of
  multiple reference sentences for each source. We present baseline results from
  training a Neural Machine Translation system and fine-tuning existing
  multilingual translation models. Our findings suggest that spelling
  inconsistencies remain a challenge in Emakhuwa. Additionally, the baseline
  models underperformed on this evaluation set, underscoring the necessity for
  further research to enhance machine translation quality for Emakhuwa. The data
  is publicly available at https://huggingface.co/datasets/LIACC/Emakhuwa-FLORES.