MURI: High-Quality Instruction Tuning Datasets for Low-Resource
  Languages via Reverse Instructions
Instruction tuning enhances large language models (LLMs) by aligning them
with human preferences across diverse tasks. Traditional approaches to create
instruction tuning datasets face serious challenges for low-resource languages
due to their dependence on data annotation. This work introduces a novel
method, Multilingual Reverse Instructions (MURI), which generates high-quality
instruction tuning datasets for low-resource languages without requiring human
annotators or pre-existing multilingual models. Utilizing reverse instructions
and a translation pipeline, MURI produces instruction-output pairs from
existing human-written texts in low-resource languages. This method ensures
cultural relevance and diversity by sourcing texts from different native
domains and applying filters to eliminate inappropriate content. Our dataset,
MURI-IT, includes more than 2 million instruction-output pairs across 200
languages. Evaluation by native speakers and fine-tuning experiments with mT5
models demonstrate the approach's effectiveness for both NLU and open-ended
generation. We publicly release datasets and models at
https://github.com/akoksal/muri.
66ecd2dc03b738ba70ade144
Abdullatif Köksal
admin_assigned
2024-09-20T09:26:25.888Z
5e089415fcf41d740b69964e
/avatars/dae50bd356fd44fe903c113441231b66.svg
Abdullatif Koksal
user
akoksal
66ecd2dc03b738ba70ade145
Marion Thaler
admin_assigned
2024-09-20T09:26:31.645Z
64b3c5104c3cc95a755ecd41
/avatars/c541717f4ff8c4ec535126e3ec4e5c00.svg
Marion Thaler
user
noriamt
66ecd2dc03b738ba70ade146
Ayyoob Imani
66ecd2dc03b738ba70ade147
Ahmet Üstün
admin_assigned
2024-09-20T09:27:41.491Z
60d35d7ad7b174177faabd5b
/avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
Ahmet Üstün
user
ahmetu
66ecd2dc03b738ba70ade148
Anna Korhonen
admin_assigned
2024-09-20T09:27:47.438Z
617a6284941993035fbaf299
https://cdn-avatars.huggingface.co/v1/production/uploads/1635410461794-noauth.jpeg
Anna Korhonen
user
akorhonen
66ecd2dc03b738ba70ade149
Hinrich Schütze
MURI: High-Quality Instruction Tuning Datasets for Low-Resource
  Languages via Reverse Instructions
Instruction tuning enhances large language models (LLMs) by aligning them
with human preferences across diverse tasks. Traditional approaches to create
instruction tuning datasets face serious challenges for low-resource languages
due to their dependence on data annotation. This work introduces a novel
method, Multilingual Reverse Instructions (MURI), which generates high-quality
instruction tuning datasets for low-resource languages without requiring human
annotators or pre-existing multilingual models. Utilizing reverse instructions
and a translation pipeline, MURI produces instruction-output pairs from
existing human-written texts in low-resource languages. This method ensures
cultural relevance and diversity by sourcing texts from different native
domains and applying filters to eliminate inappropriate content. Our dataset,
MURI-IT, includes more than 2 million instruction-output pairs across 200
languages. Evaluation by native speakers and fine-tuning experiments with mT5
models demonstrate the approach's effectiveness for both NLU and open-ended
generation. We publicly release datasets and models at
https://github.com/akoksal/muri.
MURI generates instruction tuning datasets for low-resource languages without human annotators, improving model performance in NLU and open-ended generation.
Multilingual Reverse Instructions (MURI) instruction tuning low-resource languages reverse instructions translation pipeline instruction-output pairs cultural relevance mT5 models NLU