title: Kanana: Compute-efficient Bilingual Language Models
summary: We introduce Kanana, a series of bilingual language models that demonstrate
  exceeding performance in Korean and competitive performance in English. The
  computational cost of Kanana is significantly lower than that of
  state-of-the-art models of similar size. The report details the techniques
  employed during pre-training to achieve compute-efficient yet competitive
  models, including high quality data filtering, staged pre-training, depth
  up-scaling, and pruning and distillation. Furthermore, the report outlines the
  methodologies utilized during the post-training of the Kanana models,
  encompassing supervised fine-tuning and preference optimization, aimed at
  enhancing their capability for seamless interaction with users. Lastly, the
  report elaborates on plausible approaches used for language model adaptation to
  specific scenarios, such as embedding, retrieval augmented generation, and
  function calling. The Kanana model series spans from 2.1B to 32.5B parameters
  with 2.1B models (base, instruct, embedding) publicly released to promote
  research on Korean language models.
paper_authors._id: 67bfe1bf4426925c82fe5953
paper_authors.name: Kanana LLM Team
paper_authors._id: 67bfe1bf4426925c82fe5954
paper_authors.name: Yunju Bak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-27T12:55:35.505Z
paper_authors._id: 64d08bd75de9e1e911b24226
paper_authors.avatarUrl: /avatars/e572bb47659393573a0c1fb3d333dd7b.svg
paper_authors.fullname: Yunju Bak
paper_authors.type: user
paper_authors.user: yunjubak63
paper_authors._id: 67bfe1bf4426925c82fe5955
paper_authors.name: Hojin Lee
paper_authors._id: 67bfe1bf4426925c82fe5956
paper_authors.name: Minho Ryu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:17.979Z
paper_authors._id: 60436d159e905013ae8715d7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1623809612769-60436d159e905013ae8715d7.jpeg
paper_authors.fullname: Minho Ryu
paper_authors.type: user
paper_authors.user: bzantium
paper_authors._id: 67bfe1bf4426925c82fe5957
paper_authors.name: Jiyeon Ham
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:11.786Z
paper_authors._id: 66ebb4fdc5b2c25450fd17de
paper_authors.avatarUrl: /avatars/e6b40dcbe2eba838ba21be9221758a3c.svg
paper_authors.fullname: Jiyeon Ham
paper_authors.type: user
paper_authors.user: jiyeonham
paper_authors._id: 67bfe1bf4426925c82fe5958
paper_authors.name: Seungjae Jung
paper_authors._id: 67bfe1bf4426925c82fe5959
paper_authors.name: Daniel Wontae Nam
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:09.613Z
paper_authors._id: 66c82a50c1b3c03c61aea140
paper_authors.avatarUrl: /avatars/3c508f96bdca2f2ce9746d3decd4718e.svg
paper_authors.fullname: daniel nam
paper_authors.type: user
paper_authors.user: daniel-rl2
paper_authors._id: 67bfe1bf4426925c82fe595a
paper_authors.name: Taegyeong Eo
paper_authors._id: 67bfe1bf4426925c82fe595b
paper_authors.name: Donghun Lee
paper_authors._id: 67bfe1bf4426925c82fe595c
paper_authors.name: Doohae Jung
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:06.858Z
paper_authors._id: 6142e17fe9e656d4459121e4
paper_authors.avatarUrl: /avatars/6baebd4598a845ec7fdb735eb0d53139.svg
paper_authors.fullname: Doohae Jung
paper_authors.type: user
paper_authors.user: Doohae
paper_authors._id: 67bfe1bf4426925c82fe595d
paper_authors.name: Boseop Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:01.989Z
paper_authors._id: 60f559be68ee3ef098e407cf
paper_authors.avatarUrl: /avatars/e1f00ff1c1c9fa7f591535d39c7d5e44.svg
paper_authors.fullname: Boseop Kim
paper_authors.type: user
paper_authors.user: seopbo
paper_authors._id: 67bfe1bf4426925c82fe595e
paper_authors.name: Nayeon Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:13.867Z
paper_authors._id: 6605028007a154c768e1c4c7
paper_authors.avatarUrl: /avatars/88678edb83fdb466067e38acd22d07de.svg
paper_authors.fullname: Nayeon Kim
paper_authors.type: user
paper_authors.user: lana-ny
paper_authors._id: 67bfe1bf4426925c82fe595f
paper_authors.name: Jaesun Park
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:15.898Z
paper_authors._id: 6136f65440e43b8f748a0833
paper_authors.avatarUrl: /avatars/f72a5ae3d3e94485de8aed8df94abdad.svg
paper_authors.fullname: Jaesun Park
paper_authors.type: user
paper_authors.user: jaesun
paper_authors._id: 67bfe1bf4426925c82fe5960
paper_authors.name: Hyunho Kim
paper_authors._id: 67bfe1bf4426925c82fe5961
paper_authors.name: Hyunwoong Ko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-27T12:58:05.546Z
paper_authors._id: 5fd888cf61e46993190ce543
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634604273263-5fd888cf61e46993190ce543.jpeg
paper_authors.fullname: Hyunwoong Ko
paper_authors.type: user
paper_authors.user: hyunwoongko
paper_authors._id: 67bfe1bf4426925c82fe5962
paper_authors.name: Changmin Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:14:04.506Z
paper_authors._id: 63d268bb57ab367124ea7b75
paper_authors.avatarUrl: /avatars/11312cde1e9f077aa9e5103b48be5de6.svg
paper_authors.fullname: Changmin Lee
paper_authors.type: user
paper_authors.user: changminlee
paper_authors._id: 67bfe1bf4426925c82fe5963
paper_authors.name: Kyoung-Woon On
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-27T12:58:11.269Z
paper_authors._id: 62bd31e1d2c8a6542f53fcba
paper_authors.avatarUrl: /avatars/4ac18a7bcaf9dd3885b0478dea90818f.svg
paper_authors.fullname: Kyoung-Woon On
paper_authors.type: user
paper_authors.user: kloud
paper_authors._id: 67bfe1bf4426925c82fe5964
paper_authors.name: Seulye Baeg
paper_authors._id: 67bfe1bf4426925c82fe5965
paper_authors.name: Junrae Cho
paper_authors._id: 67bfe1bf4426925c82fe5966
paper_authors.name: Sunghee Jung
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-03T08:29:34.214Z
paper_authors._id: 65e30342e8b017ee1384824c
paper_authors.avatarUrl: /avatars/e5d07b037f611ccfaf719959d971d102.svg
paper_authors.fullname: Sunghee Jung
paper_authors.type: user
paper_authors.user: hash2430
paper_authors._id: 67bfe1bf4426925c82fe5967
paper_authors.name: Jieun Kang
paper_authors._id: 67bfe1bf4426925c82fe5968
paper_authors.name: EungGyun Kim
paper_authors._id: 67bfe1bf4426925c82fe5969
paper_authors.name: Eunhwa Kim
paper_authors._id: 67bfe1bf4426925c82fe596a
paper_authors.name: Byeongil Ko
paper_authors._id: 67bfe1bf4426925c82fe596b
paper_authors.name: Daniel Lee
paper_authors._id: 67bfe1bf4426925c82fe596c
paper_authors.name: Minchul Lee
paper_authors._id: 67bfe1bf4426925c82fe596d
paper_authors.name: Miok Lee
paper_authors._id: 67bfe1bf4426925c82fe596e
paper_authors.name: Shinbok Lee
paper_authors._id: 67bfe1bf4426925c82fe596f
paper_authors.name: Gaeun Seo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-27T12:59:39.670Z
paper_authors._id: 63148a8f5f47a18962765802
paper_authors.avatarUrl: /avatars/bc58a863727794006dddf758efa09411.svg
paper_authors.fullname: gaeunseo
paper_authors.type: user
paper_authors.user: gaeunseo
paper_title: Kanana: Compute-efficient Bilingual Language Models
paper_summary: We introduce Kanana, a series of bilingual language models that demonstrate
  exceeding performance in Korean and competitive performance in English. The
  computational cost of Kanana is significantly lower than that of
  state-of-the-art models of similar size. The report details the techniques
  employed during pre-training to achieve compute-efficient yet competitive
  models, including high quality data filtering, staged pre-training, depth
  up-scaling, and pruning and distillation. Furthermore, the report outlines the
  methodologies utilized during the post-training of the Kanana models,
  encompassing supervised fine-tuning and preference optimization, aimed at
  enhancing their capability for seamless interaction with users. Lastly, the
  report elaborates on plausible approaches used for language model adaptation to
  specific scenarios, such as embedding, retrieval augmented generation, and
  function calling. The Kanana model series spans from 2.1B to 32.5B parameters
  with 2.1B models (base, instruct, embedding) publicly released to promote
  research on Korean language models.
paper_ai_summary: Kanana, a series of bilingual language models, achieves superior performance in Korean and competitive performance in English with lower computational costs through efficient pre-training and post-training techniques.
paper_ai_keywords: high quality data filtering staged pre-training depth up-scaling pruning distillation supervised fine-tuning preference optimization embedding retrieval augmented generation function calling