title: Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan
    Arabic Dialect
summary: We introduce Atlas-Chat, the first-ever collection of large language models
  specifically developed for dialectal Arabic. Focusing on Moroccan Arabic, also
  known as Darija, we construct our instruction dataset by consolidating existing
  Darija language resources, creating novel datasets both manually and
  synthetically, and translating English instructions with stringent quality
  control. Atlas-Chat-9B and 2B models, fine-tuned on the dataset, exhibit
  superior ability in following Darija instructions and performing standard NLP
  tasks. Notably, our models outperform both state-of-the-art and
  Arabic-specialized LLMs like LLaMa, Jais, and AceGPT, e.g., achieving a 13%
  performance boost over a larger 13B model on DarijaMMLU, in our newly
  introduced evaluation suite for Darija covering both discriminative and
  generative tasks. Furthermore, we perform an experimental analysis of various
  fine-tuning strategies and base model choices to determine optimal
  configurations. All our resources are publicly accessible, and we believe our
  work offers comprehensive design methodologies of instruction-tuning for
  low-resource language variants, which are often neglected in favor of data-rich
  languages by contemporary LLMs.
paper_authors._id: 66f650dfb35008620e600b6f
paper_authors.name: Guokan Shang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-27T13:47:58.578Z
paper_authors._id: 6087e598e2b7cc3a117b0dc5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6087e598e2b7cc3a117b0dc5/Ctz_W-uo1gOQRBHXalD1P.png
paper_authors.fullname: Guokan Shang
paper_authors.type: user
paper_authors.user: guokan-shang
paper_authors._id: 66f650dfb35008620e600b70
paper_authors.name: Hadi Abdine
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-27T06:57:58.895Z
paper_authors._id: 66448b4fecac3bc79b26304f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66448b4fecac3bc79b26304f/aPH3UFbc20CL2Bz2yn7nH.jpeg
paper_authors.fullname: Hadi Abdine
paper_authors.type: user
paper_authors.user: habdine
paper_authors._id: 66f650dfb35008620e600b71
paper_authors.name: Yousef Khoubrane
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-30T08:07:07.298Z
paper_authors._id: 6617fd349d3673a2621d6ea5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6617fd349d3673a2621d6ea5/-PXtZY3y2zWbaRx9qFc39.jpeg
paper_authors.fullname: Yousef Khoubrane
paper_authors.type: user
paper_authors.user: khoubrane-yousef
paper_authors._id: 66f650dfb35008620e600b72
paper_authors.name: Amr Mohamed
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-30T12:08:31.573Z
paper_authors._id: 655efd24afee0e00788bb589
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/655efd24afee0e00788bb589/22guLxIWNybbJR3jI-c4w.jpeg
paper_authors.fullname: Amr Mohamed
paper_authors.type: user
paper_authors.user: amr-mohamed
paper_authors._id: 66f650dfb35008620e600b73
paper_authors.name: Yassine Abbahaddou
paper_authors._id: 66f650dfb35008620e600b74
paper_authors.name: Sofiane Ennadir
paper_authors._id: 66f650dfb35008620e600b75
paper_authors.name: Imane Momayiz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-02T08:44:58.056Z
paper_authors._id: 64f78a17ceabf1e6fc352fbc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f78a17ceabf1e6fc352fbc/KFEGWmhC7PGNwbVzqoEJB.jpeg
paper_authors.fullname: Imane Momayiz
paper_authors.type: user
paper_authors.user: imomayiz
paper_authors._id: 66f650dfb35008620e600b76
paper_authors.name: Xuguang Ren
paper_authors._id: 66f650dfb35008620e600b77
paper_authors.name: Eric Moulines
paper_authors._id: 66f650dfb35008620e600b78
paper_authors.name: Preslav Nakov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-02T08:43:45.303Z
paper_authors._id: 647f7eb25e1bc4753746bf9f
paper_authors.avatarUrl: /avatars/cc9c6210fdc822d8a106937e747dff41.svg
paper_authors.fullname: Preslav Nakov
paper_authors.type: user
paper_authors.user: preslavnakov
paper_authors._id: 66f650dfb35008620e600b79
paper_authors.name: Michalis Vazirgiannis
paper_authors._id: 66f650dfb35008620e600b7a
paper_authors.name: Eric Xing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-02T08:44:14.611Z
paper_authors._id: 64ff67722ad36636be6c4542
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sLIrNelAWPVOy4e3oo5LB.jpeg
paper_authors.fullname: Eric Xing
paper_authors.type: user
paper_authors.user: EricX003
paper_title: Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan
    Arabic Dialect
paper_summary: We introduce Atlas-Chat, the first-ever collection of large language models
  specifically developed for dialectal Arabic. Focusing on Moroccan Arabic, also
  known as Darija, we construct our instruction dataset by consolidating existing
  Darija language resources, creating novel datasets both manually and
  synthetically, and translating English instructions with stringent quality
  control. Atlas-Chat-9B and 2B models, fine-tuned on the dataset, exhibit
  superior ability in following Darija instructions and performing standard NLP
  tasks. Notably, our models outperform both state-of-the-art and
  Arabic-specialized LLMs like LLaMa, Jais, and AceGPT, e.g., achieving a 13%
  performance boost over a larger 13B model on DarijaMMLU, in our newly
  introduced evaluation suite for Darija covering both discriminative and
  generative tasks. Furthermore, we perform an experimental analysis of various
  fine-tuning strategies and base model choices to determine optimal
  configurations. All our resources are publicly accessible, and we believe our
  work offers comprehensive design methodologies of instruction-tuning for
  low-resource language variants, which are often neglected in favor of data-rich
  languages by contemporary LLMs.
paper_ai_summary: Atlas-Chat, a collection of large language models fine-tuned for Moroccan Arabic (Darija), outperforms state-of-the-art LLMs on Darija-specific tasks.
paper_ai_keywords: large language models dialectal Arabic Moroccan Arabic Darija instruction dataset fine-tuned NLP tasks LLaMa Jais AceGPT DarijaMMLU fine-tuning strategies base model choices instruction-tuning low-resource language variants