title: InkubaLM: A small language model for low-resource African languages
summary: High-resource language models often fall short in the African context, where
  there is a critical need for models that are efficient, accessible, and locally
  relevant, even amidst significant computing and data constraints. This paper
  introduces InkubaLM, a small language model with 0.4 billion parameters, which
  achieves performance comparable to models with significantly larger parameter
  counts and more extensive training data on tasks such as machine translation,
  question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
  outperforms many larger models in sentiment analysis and demonstrates
  remarkable consistency across multiple languages. This work represents a
  pivotal advancement in challenging the conventional paradigm that effective
  language models must rely on substantial resources. Our model and datasets are
  publicly available \url{https://huggingface.co/lelapa} to encourage
  research and development on low-resource languages.
paper_authors._id: 66d585b72e0412fa2abab646
paper_authors.name: Atnafu Lambebo Tonja
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:20.105Z
paper_authors._id: 6245232cf59b8d262df30db6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6245232cf59b8d262df30db6/zmQuv1vqGL3MsCAnzGm9y.jpeg
paper_authors.fullname: Atnafu Lambebo Tonja
paper_authors.type: user
paper_authors.user: Atnafu
paper_authors._id: 66d585b72e0412fa2abab647
paper_authors.name: Bonaventure F. P. Dossou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:29.734Z
paper_authors._id: 605dd96184a51781b143a131
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/605dd96184a51781b143a131/wBUY_uAcQOWdRW21uZLS7.jpeg
paper_authors.fullname: Bonaventure Dossou
paper_authors.type: user
paper_authors.user: bonadossou
paper_authors._id: 66d585b72e0412fa2abab648
paper_authors.name: Jessica Ojo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:35.872Z
paper_authors._id: 650030261e170b85743bf3ca
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650030261e170b85743bf3ca/3iq_cmj7wq03OgAR0CTtE.jpeg
paper_authors.fullname: Jessica O. Ojo
paper_authors.type: user
paper_authors.user: JessicaOjo
paper_authors._id: 66d585b72e0412fa2abab649
paper_authors.name: Jenalea Rajab
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:41.768Z
paper_authors._id: 64490668638bfa3ce138f373
paper_authors.avatarUrl: /avatars/7db8f63634959927443925f49368ed5e.svg
paper_authors.fullname: Jenalea Rajab
paper_authors.type: user
paper_authors.user: Jenalea
paper_authors._id: 66d585b72e0412fa2abab64a
paper_authors.name: Fadel Thior
paper_authors._id: 66d585b72e0412fa2abab64b
paper_authors.name: Eric Peter Wairagala
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:51.410Z
paper_authors._id: 6151c7140d6899cb526dbd96
paper_authors.avatarUrl: /avatars/1fc151e025839e92af0beacd5a68ebff.svg
paper_authors.fullname: Eric Peter Wairagala
paper_authors.type: user
paper_authors.user: EricPeter
paper_authors._id: 66d585b72e0412fa2abab64c
paper_authors.name: Aremu Anuoluwapo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:00:58.957Z
paper_authors._id: 628a9a8b6a2a449b99aa1969
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1653342087811-628a9a8b6a2a449b99aa1969.jpeg
paper_authors.fullname: Aremu Anuoluwapo
paper_authors.type: user
paper_authors.user: aremuadeolajr
paper_authors._id: 66d585b72e0412fa2abab64d
paper_authors.name: Pelonomi Moiloa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:03.996Z
paper_authors._id: 6481b502ec65b8b77d85bca5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6481b502ec65b8b77d85bca5/D0AwZ-9znmUHb-myzHL3M.jpeg
paper_authors.fullname: Pelonomi Moiloa
paper_authors.type: user
paper_authors.user: pelonomi
paper_authors._id: 66d585b72e0412fa2abab64e
paper_authors.name: Jade Abbott
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:09.828Z
paper_authors._id: 5fbf94dcf6c5c10afc0c9e09
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1606390935561-noauth.jpeg
paper_authors.fullname: Jade Abbott
paper_authors.type: user
paper_authors.user: alienelf
paper_authors._id: 66d585b72e0412fa2abab64f
paper_authors.name: Vukosi Marivate
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-02T10:01:15.372Z
paper_authors._id: 617a68fef53050061a7294ea
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/617a68fef53050061a7294ea/J9B1gZ9CUPRMahP6QDSPL.jpeg
paper_authors.fullname: Vukosi Marivate
paper_authors.type: user
paper_authors.user: vukosi
paper_authors._id: 66d585b72e0412fa2abab650
paper_authors.name: Benjamin Rosman
paper_title: InkubaLM: A small language model for low-resource African languages
paper_summary: High-resource language models often fall short in the African context, where
  there is a critical need for models that are efficient, accessible, and locally
  relevant, even amidst significant computing and data constraints. This paper
  introduces InkubaLM, a small language model with 0.4 billion parameters, which
  achieves performance comparable to models with significantly larger parameter
  counts and more extensive training data on tasks such as machine translation,
  question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
  outperforms many larger models in sentiment analysis and demonstrates
  remarkable consistency across multiple languages. This work represents a
  pivotal advancement in challenging the conventional paradigm that effective
  language models must rely on substantial resources. Our model and datasets are
  publicly available \url{https://huggingface.co/lelapa} to encourage
  research and development on low-resource languages.
paper_ai_summary: InkubaLM, a small language model with 0.4 billion parameters, outperforms larger models in various tasks including sentiment analysis and demonstrates remarkable consistency across multiple languages, challenging the necessity for substantial resources in effective language models.