InkubaLM: A small language model for low-resource African languages
High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.
66d585b72e0412fa2abab646
Atnafu Lambebo Tonja
admin_assigned
2024-09-02T10:00:20.105Z
6245232cf59b8d262df30db6
https://cdn-avatars.huggingface.co/v1/production/uploads/6245232cf59b8d262df30db6/zmQuv1vqGL3MsCAnzGm9y.jpeg
Atnafu Lambebo Tonja
user
Atnafu
66d585b72e0412fa2abab647
Bonaventure F. P. Dossou
admin_assigned
2024-09-02T10:00:29.734Z
605dd96184a51781b143a131
https://cdn-avatars.huggingface.co/v1/production/uploads/605dd96184a51781b143a131/wBUY_uAcQOWdRW21uZLS7.jpeg
Bonaventure Dossou
user
bonadossou
66d585b72e0412fa2abab648
Jessica Ojo
admin_assigned
2024-09-02T10:00:35.872Z
650030261e170b85743bf3ca
https://cdn-avatars.huggingface.co/v1/production/uploads/650030261e170b85743bf3ca/3iq_cmj7wq03OgAR0CTtE.jpeg
Jessica O. Ojo
user
JessicaOjo
66d585b72e0412fa2abab649
Jenalea Rajab
admin_assigned
2024-09-02T10:00:41.768Z
64490668638bfa3ce138f373
/avatars/7db8f63634959927443925f49368ed5e.svg
Jenalea Rajab
user
Jenalea
66d585b72e0412fa2abab64a
Fadel Thior
66d585b72e0412fa2abab64b
Eric Peter Wairagala
admin_assigned
2024-09-02T10:00:51.410Z
6151c7140d6899cb526dbd96
/avatars/1fc151e025839e92af0beacd5a68ebff.svg
Eric Peter Wairagala
user
EricPeter
66d585b72e0412fa2abab64c
Aremu Anuoluwapo
admin_assigned
2024-09-02T10:00:58.957Z
628a9a8b6a2a449b99aa1969
https://cdn-avatars.huggingface.co/v1/production/uploads/1653342087811-628a9a8b6a2a449b99aa1969.jpeg
Aremu Anuoluwapo
user
aremuadeolajr
66d585b72e0412fa2abab64d
Pelonomi Moiloa
admin_assigned
2024-09-02T10:01:03.996Z
6481b502ec65b8b77d85bca5
https://cdn-avatars.huggingface.co/v1/production/uploads/6481b502ec65b8b77d85bca5/D0AwZ-9znmUHb-myzHL3M.jpeg
Pelonomi Moiloa
user
pelonomi
66d585b72e0412fa2abab64e
Jade Abbott
admin_assigned
2024-09-02T10:01:09.828Z
5fbf94dcf6c5c10afc0c9e09
https://cdn-avatars.huggingface.co/v1/production/uploads/1606390935561-noauth.jpeg
Jade Abbott
user
alienelf
66d585b72e0412fa2abab64f
Vukosi Marivate
admin_assigned
2024-09-02T10:01:15.372Z
617a68fef53050061a7294ea
https://cdn-avatars.huggingface.co/v1/production/uploads/617a68fef53050061a7294ea/J9B1gZ9CUPRMahP6QDSPL.jpeg
Vukosi Marivate
user
vukosi
66d585b72e0412fa2abab650
Benjamin Rosman
InkubaLM: A small language model for low-resource African languages
High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.
InkubaLM, a small language model with 0.4 billion parameters, outperforms larger models in various tasks including sentiment analysis and demonstrates remarkable consistency across multiple languages, challenging the necessity for substantial resources in effective language models.