title: InkubaLM: A small language model for low-resource African languages
summary: High-resource language models often fall short in the African context, where
  there is a critical need for models that are efficient, accessible, and locally
  relevant, even amidst significant computing and data constraints. This paper
  introduces InkubaLM, a small language model with 0.4 billion parameters, which
  achieves performance comparable to models with significantly larger parameter
  counts and more extensive training data on tasks such as machine translation,
  question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
  outperforms many larger models in sentiment analysis and demonstrates
  remarkable consistency across multiple languages. This work represents a
  pivotal advancement in challenging the conventional paradigm that effective
  language models must rely on substantial resources. Our model and datasets are
  publicly available \url{https://huggingface.co/lelapa} to encourage
  research and development on low-resource languages.
paper_authors._id: 66d585b72e0412fa2abab646
paper_authors.name: Atnafu Lambebo Tonja
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:20.105Z
paper_authors._id: 6245232cf59b8d262df30db6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6245232cf59b8d262df30db6/zmQuv1vqGL3MsCAnzGm9y.jpeg
paper_authors.fullname: Atnafu Lambebo Tonja
paper_authors.type: user
paper_authors.user: Atnafu
paper_authors._id: 66d585b72e0412fa2abab647
paper_authors.name: Bonaventure F. P. Dossou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:29.734Z
paper_authors._id: 605dd96184a51781b143a131
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/605dd96184a51781b143a131/wBUY_uAcQOWdRW21uZLS7.jpeg
paper_authors.fullname: Bonaventure Dossou
paper_authors.type: user
paper_authors.user: bonadossou
paper_authors._id: 66d585b72e0412fa2abab648
paper_authors.name: Jessica Ojo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:35.872Z
paper_authors._id: 650030261e170b85743bf3ca
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650030261e170b85743bf3ca/3iq_cmj7wq03OgAR0CTtE.jpeg
paper_authors.fullname: Jessica O. Ojo
paper_authors.type: user
paper_authors.user: JessicaOjo
paper_authors._id: 66d585b72e0412fa2abab649
paper_authors.name: Jenalea Rajab
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:41.768Z
paper_authors._id: 64490668638bfa3ce138f373
paper_authors.avatarUrl: /avatars/7db8f63634959927443925f49368ed5e.svg
paper_authors.fullname: Jenalea Rajab
paper_authors.type: user
paper_authors.user: Jenalea
paper_authors._id: 66d585b72e0412fa2abab64a
paper_authors.name: Fadel Thior
paper_authors._id: 66d585b72e0412fa2abab64b
paper_authors.name: Eric Peter Wairagala
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:51.410Z
paper_authors._id: 6151c7140d6899cb526dbd96
paper_authors.avatarUrl: /avatars/1fc151e025839e92af0beacd5a68ebff.svg
paper_authors.fullname: Eric Peter Wairagala
paper_authors.type: user
paper_authors.user: EricPeter
paper_authors._id: 66d585b72e0412fa2abab64c
paper_authors.name: Aremu Anuoluwapo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:58.957Z
paper_authors._id: 628a9a8b6a2a449b99aa1969
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1653342087811-628a9a8b6a2a449b99aa1969.jpeg
paper_authors.fullname: Aremu Anuoluwapo
paper_authors.type: user
paper_authors.user: aremuadeolajr
paper_authors._id: 66d585b72e0412fa2abab64d
paper_authors.name: Pelonomi Moiloa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:03.996Z
paper_authors._id: 6481b502ec65b8b77d85bca5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6481b502ec65b8b77d85bca5/D0AwZ-9znmUHb-myzHL3M.jpeg
paper_authors.fullname: Pelonomi Moiloa
paper_authors.type: user
paper_authors.user: pelonomi
paper_authors._id: 66d585b72e0412fa2abab64e
paper_authors.name: Jade Abbott
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:09.828Z
paper_authors._id: 5fbf94dcf6c5c10afc0c9e09
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1606390935561-noauth.jpeg
paper_authors.fullname: Jade Abbott
paper_authors.type: user
paper_authors.user: alienelf
paper_authors._id: 66d585b72e0412fa2abab64f
paper_authors.name: Vukosi Marivate
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:15.372Z
paper_authors._id: 617a68fef53050061a7294ea
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/617a68fef53050061a7294ea/J9B1gZ9CUPRMahP6QDSPL.jpeg
paper_authors.fullname: Vukosi Marivate
paper_authors.type: user
paper_authors.user: vukosi
paper_authors._id: 66d585b72e0412fa2abab650
paper_authors.name: Benjamin Rosman
paper_title: InkubaLM: A small language model for low-resource African languages
paper_summary: High-resource language models often fall short in the African context, where
  there is a critical need for models that are efficient, accessible, and locally
  relevant, even amidst significant computing and data constraints. This paper
  introduces InkubaLM, a small language model with 0.4 billion parameters, which
  achieves performance comparable to models with significantly larger parameter
  counts and more extensive training data on tasks such as machine translation,
  question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
  outperforms many larger models in sentiment analysis and demonstrates
  remarkable consistency across multiple languages. This work represents a
  pivotal advancement in challenging the conventional paradigm that effective
  language models must rely on substantial resources. Our model and datasets are
  publicly available \url{https://huggingface.co/lelapa} to encourage
  research and development on low-resource languages.
paper_ai_keywords: small language model InkubaLM 0.4 billion parameters machine translation question-answering AfriMMLU AfriXnli sentiment analysis low-resource languages