title: MURI: High-Quality Instruction Tuning Datasets for Low-Resource
    Languages via Reverse Instructions
summary: Instruction tuning enhances large language models (LLMs) by aligning them
  with human preferences across diverse tasks. Traditional approaches to create
  instruction tuning datasets face serious challenges for low-resource languages
  due to their dependence on data annotation. This work introduces a novel
  method, Multilingual Reverse Instructions (MURI), which generates high-quality
  instruction tuning datasets for low-resource languages without requiring human
  annotators or pre-existing multilingual models. Utilizing reverse instructions
  and a translation pipeline, MURI produces instruction-output pairs from
  existing human-written texts in low-resource languages. This method ensures
  cultural relevance and diversity by sourcing texts from different native
  domains and applying filters to eliminate inappropriate content. Our dataset,
  MURI-IT, includes more than 2 million instruction-output pairs across 200
  languages. Evaluation by native speakers and fine-tuning experiments with mT5
  models demonstrate the approach's effectiveness for both NLU and open-ended
  generation. We publicly release datasets and models at
  https://github.com/akoksal/muri.
paper_authors._id: 66ecd2dc03b738ba70ade144
paper_authors.name: Abdullatif Köksal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:26:25.888Z
paper_authors._id: 5e089415fcf41d740b69964e
paper_authors.avatarUrl: /avatars/dae50bd356fd44fe903c113441231b66.svg
paper_authors.fullname: Abdullatif Koksal
paper_authors.type: user
paper_authors.user: akoksal
paper_authors._id: 66ecd2dc03b738ba70ade145
paper_authors.name: Marion Thaler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:26:31.645Z
paper_authors._id: 64b3c5104c3cc95a755ecd41
paper_authors.avatarUrl: /avatars/c541717f4ff8c4ec535126e3ec4e5c00.svg
paper_authors.fullname: Marion Thaler
paper_authors.type: user
paper_authors.user: noriamt
paper_authors._id: 66ecd2dc03b738ba70ade146
paper_authors.name: Ayyoob Imani
paper_authors._id: 66ecd2dc03b738ba70ade147
paper_authors.name: Ahmet Üstün
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:27:41.491Z
paper_authors._id: 60d35d7ad7b174177faabd5b
paper_authors.avatarUrl: /avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
paper_authors.fullname: Ahmet Üstün
paper_authors.type: user
paper_authors.user: ahmetu
paper_authors._id: 66ecd2dc03b738ba70ade148
paper_authors.name: Anna Korhonen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:27:47.438Z
paper_authors._id: 617a6284941993035fbaf299
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1635410461794-noauth.jpeg
paper_authors.fullname: Anna Korhonen
paper_authors.type: user
paper_authors.user: akorhonen
paper_authors._id: 66ecd2dc03b738ba70ade149
paper_authors.name: Hinrich Schütze
paper_title: MURI: High-Quality Instruction Tuning Datasets for Low-Resource
    Languages via Reverse Instructions
paper_summary: Instruction tuning enhances large language models (LLMs) by aligning them
  with human preferences across diverse tasks. Traditional approaches to create
  instruction tuning datasets face serious challenges for low-resource languages
  due to their dependence on data annotation. This work introduces a novel
  method, Multilingual Reverse Instructions (MURI), which generates high-quality
  instruction tuning datasets for low-resource languages without requiring human
  annotators or pre-existing multilingual models. Utilizing reverse instructions
  and a translation pipeline, MURI produces instruction-output pairs from
  existing human-written texts in low-resource languages. This method ensures
  cultural relevance and diversity by sourcing texts from different native
  domains and applying filters to eliminate inappropriate content. Our dataset,
  MURI-IT, includes more than 2 million instruction-output pairs across 200
  languages. Evaluation by native speakers and fine-tuning experiments with mT5
  models demonstrate the approach's effectiveness for both NLU and open-ended
  generation. We publicly release datasets and models at
  https://github.com/akoksal/muri.
paper_ai_summary: MURI generates instruction tuning datasets for low-resource languages without human annotators, improving model performance in NLU and open-ended generation.
paper_ai_keywords: Multilingual Reverse Instructions (MURI) instruction tuning low-resource languages reverse instructions translation pipeline instruction-output pairs cultural relevance mT5 models NLU