Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara
<|reference_start|>Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara: In contexts with limited computational and data resources, high-resource language models often prove inadequate, particularly when addressing the specific needs of Malay languages. This paper introduces a Personal Intelligence System designed to efficiently integrate both on-device and server-based models. The system incorporates SLiM-34M for on-device processing, optimized for low memory and power usage, and MANYAK-1.3B for server-based tasks, allowing for scalable, high-performance language processing. The models achieve significant results across various tasks, such as machine translation, question-answering, and translate IndoMMLU. Particularly noteworthy is SLiM-34M's ability to achieve a high improvement in accuracy compared to other LLMs while using 2 times fewer pre-training tokens. This work challenges the prevailing assumption that large-scale computational resources are necessary to build effective language models, contributing to the development of resource-efficient models for the Malay language with the unique orchestration between SLiM-34M and MANYAK-1.3B.<|reference_end|>
@article{nazri2024personal,
    title={Personal Intelligence System UniLM: Hybrid On-Device Small Language
  Model and Server-Based Large Language Model for Malay Nusantara},
    author={Azree Nazri, Olalekan Agbolade, Faisal Aziz},
    journal={arXiv preprint arXiv:2410.06973},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.06973},
    primaryClass={cs.CL cs.AI}
}