title: SeaLLMs -- Large Language Models for Southeast Asia
summary: Despite the remarkable achievements of large language models (LLMs) in
  various tasks, there remains a linguistic bias that favors high-resource
  languages, such as English, often at the expense of low-resource and regional
  languages. To address this imbalance, we introduce SeaLLMs, an innovative
  series of language models that specifically focuses on Southeast Asian (SEA)
  languages. SeaLLMs are built upon the Llama-2 model and further advanced
  through continued pre-training with an extended vocabulary, specialized
  instruction and alignment tuning to better capture the intricacies of regional
  languages. This allows them to respect and reflect local cultural norms,
  customs, stylistic preferences, and legal considerations. Our comprehensive
  evaluation demonstrates that SeaLLM-13b models exhibit superior performance
  across a wide spectrum of linguistic tasks and assistant-style
  instruction-following capabilities relative to comparable open-source models.
  Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai,
  Khmer, Lao, and Burmese, by large margins while remaining lightweight and
  cost-effective to operate.
paper_authors._id: 656d317b8fb1ddf0d5524830
paper_authors.name: Xuan-Phi Nguyen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-04T07:39:13.769Z
paper_authors._id: 6393f04df7e70dd0166c004e
paper_authors.avatarUrl: /avatars/acf4e9e0204a7ff7445aecc4102700cd.svg
paper_authors.fullname: Phi Nguyen
paper_authors.type: user
paper_authors.user: nxphi47
paper_authors._id: 656d317b8fb1ddf0d5524831
paper_authors.name: Wenxuan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:26:30.819Z
paper_authors._id: 60dff6ae19a362a8c27862aa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60dff6ae19a362a8c27862aa/LIYzLB3cdPh-B3XIBgBCC.jpeg
paper_authors.fullname: Wenxuan Zhang
paper_authors.type: user
paper_authors.user: isakzhang
paper_authors._id: 656d317b8fb1ddf0d5524832
paper_authors.name: Xin Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:41:36.038Z
paper_authors._id: 63913b120cf6b11c487ca31d
paper_authors.avatarUrl: /avatars/aec44edd5470dd6e767e0a25efd6fb5d.svg
paper_authors.fullname: Xin Li
paper_authors.type: user
paper_authors.user: lixin4ever
paper_authors._id: 656d317b8fb1ddf0d5524833
paper_authors.name: Mahani Aljunied
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:27:24.831Z
paper_authors._id: 6539c87ba318a98bf0d15dd8
paper_authors.avatarUrl: /avatars/beb9ba6eeacb61addc5897836bd59f55.svg
paper_authors.fullname: Mahani Aljunied
paper_authors.type: user
paper_authors.user: maljunied
paper_authors._id: 656d317b8fb1ddf0d5524834
paper_authors.name: Qingyu Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:27:32.917Z
paper_authors._id: 62bd46d35f59ff2320d7e31f
paper_authors.avatarUrl: /avatars/19b5817a5fd531cca9ea83b76f46d7d3.svg
paper_authors.fullname: Qingyu Tan
paper_authors.type: user
paper_authors.user: tonytan48
paper_authors._id: 656d317b8fb1ddf0d5524835
paper_authors.name: Liying Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:27:39.676Z
paper_authors._id: 60be079e9418b3c3717a07ce
paper_authors.avatarUrl: /avatars/2ae2822e2645f5e52be70b6f964feda3.svg
paper_authors.fullname: Liying Cheng
paper_authors.type: user
paper_authors.user: liyingnlp
paper_authors._id: 656d317b8fb1ddf0d5524836
paper_authors.name: Guanzheng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:27:50.248Z
paper_authors._id: 645475e2548f22be59847604
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645475e2548f22be59847604/EhSurrZ25u31qQ2TVXQXt.jpeg
paper_authors.fullname: Chen
paper_authors.type: user
paper_authors.user: Guanzheng
paper_authors._id: 656d317b8fb1ddf0d5524837
paper_authors.name: Yue Deng
paper_authors._id: 656d317b8fb1ddf0d5524838
paper_authors.name: Sen Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:30:32.743Z
paper_authors._id: 638dd7d8a11654155baba973
paper_authors.avatarUrl: /avatars/a95c01cd03d5da209b4d82d50aa01232.svg
paper_authors.fullname: Sen Yang
paper_authors.type: user
paper_authors.user: ringos
paper_authors._id: 656d317b8fb1ddf0d5524839
paper_authors.name: Chaoqun Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:30:39.598Z
paper_authors._id: 61657b0b20606e5e73f611cc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61657b0b20606e5e73f611cc/6ZPne2GYlWkxrx35ND1P8.png
paper_authors.fullname: CHAOQUN LIU
paper_authors.type: user
paper_authors.user: lukecq
paper_authors._id: 656d317b8fb1ddf0d552483a
paper_authors.name: Hang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:31:00.170Z
paper_authors._id: 638ee4bca26f3d5af37d7698
paper_authors.avatarUrl: /avatars/36f1c81dd82379f863601a1767ff630a.svg
paper_authors.fullname: zhanghang
paper_authors.type: user
paper_authors.user: hangzhang-nlp
paper_authors._id: 656d317b8fb1ddf0d552483b
paper_authors.name: Lidong Bing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T08:31:06.774Z
paper_authors._id: 6454685a548f22be598414c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
paper_authors.fullname: Lidong Bing
paper_authors.type: user
paper_authors.user: LidongBing
paper_title: SeaLLMs -- Large Language Models for Southeast Asia
paper_summary: Despite the remarkable achievements of large language models (LLMs) in
  various tasks, there remains a linguistic bias that favors high-resource
  languages, such as English, often at the expense of low-resource and regional
  languages. To address this imbalance, we introduce SeaLLMs, an innovative
  series of language models that specifically focuses on Southeast Asian (SEA)
  languages. SeaLLMs are built upon the Llama-2 model and further advanced
  through continued pre-training with an extended vocabulary, specialized
  instruction and alignment tuning to better capture the intricacies of regional
  languages. This allows them to respect and reflect local cultural norms,
  customs, stylistic preferences, and legal considerations. Our comprehensive
  evaluation demonstrates that SeaLLM-13b models exhibit superior performance
  across a wide spectrum of linguistic tasks and assistant-style
  instruction-following capabilities relative to comparable open-source models.
  Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai,
  Khmer, Lao, and Burmese, by large margins while remaining lightweight and
  cost-effective to operate.
paper_ai_summary: SeaLLMs, a series of Llama-2 based models tailored for Southeast Asian languages, demonstrate superior performance across linguistic tasks and outperform ChatGPT-3.5 in non-Latin languages while being lightweight and cost-effective.
paper_ai_keywords: SeaLLMs Llama-2 pre-training vocabulary instruction tuning alignment tuning linguistic tasks assistant-style ChatGPT-3.5 Thai Khmer Lao Burmese