InkubaLM: A small language model for low-resource African languages
<|reference_start|>InkubaLM: A small language model for low-resource African languages: High-resource language models often fall short in the African context, where there is a critical need for models that are efficient, accessible, and locally relevant, even amidst significant computing and data constraints. This paper introduces InkubaLM, a small language model with 0.4 billion parameters, which achieves performance comparable to models with significantly larger parameter counts and more extensive training data on tasks such as machine translation, question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM outperforms many larger models in sentiment analysis and demonstrates remarkable consistency across multiple languages. This work represents a pivotal advancement in challenging the conventional paradigm that effective language models must rely on substantial resources. Our model and datasets are publicly available at https://huggingface.co/lelapa to encourage research and development on low-resource languages.<|reference_end|>
@article{tonja2024inkubalm:,
    title={InkubaLM: A small language model for low-resource African languages},
    author={Atnafu Lambebo Tonja, Bonaventure F. P. Dossou, Jessica Ojo, Jenalea
  Rajab, Fadel Thior, Eric Peter Wairagala, Anuoluwapo Aremu, Pelonomi Moiloa,
  Jade Abbott, Vukosi Marivate, Benjamin Rosman},
    journal={arXiv preprint arXiv:2408.17024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.17024},
    primaryClass={cs.CL}
}