title: A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean
    Language Models
summary: Polyglot is a pioneering project aimed at enhancing the non-English language
  performance of multilingual language models. Despite the availability of
  various multilingual models such as mBERT (Devlin et al., 2019), XGLM (Lin et
  al., 2022), and BLOOM (Scao et al., 2022), researchers and developers often
  resort to building monolingual models in their respective languages due to the
  dissatisfaction with the current multilingual models non-English language
  capabilities. Addressing this gap, we seek to develop advanced multilingual
  language models that offer improved performance in non-English languages. In
  this paper, we introduce the Polyglot Korean models, which represent a specific
  focus rather than being multilingual in nature. In collaboration with TUNiB,
  our team collected 1.2TB of Korean data meticulously curated for our research
  journey. We made a deliberate decision to prioritize the development of Korean
  models before venturing into multilingual models. This choice was motivated by
  multiple factors: firstly, the Korean models facilitated performance
  comparisons with existing multilingual models; and finally, they catered to the
  specific needs of Korean companies and researchers. This paper presents our
  work in developing the Polyglot Korean models, which propose some steps towards
  addressing the non-English language performance gap in multilingual language
  models.
paper_authors._id: 647eaa82becb41a2729772cd
paper_authors.name: Hyunwoong Ko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-07T19:41:00.982Z
paper_authors._id: 5fd888cf61e46993190ce543
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634604273263-5fd888cf61e46993190ce543.jpeg
paper_authors.fullname: Hyunwoong Ko
paper_authors.type: user
paper_authors.user: hyunwoongko
paper_authors._id: 647eaa82becb41a2729772ce
paper_authors.name: Kichang Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-06T08:49:30.533Z
paper_authors._id: 5fa11101a13e063b8b2b5d34
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5fa11101a13e063b8b2b5d34/p2Sm4b64hn_xXg1GtrkIA.jpeg
paper_authors.fullname: Kichang Yang
paper_authors.type: user
paper_authors.user: jason9693
paper_authors._id: 647eaa82becb41a2729772cf
paper_authors.name: Minho Ryu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-06T08:50:04.739Z
paper_authors._id: 60436d159e905013ae8715d7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1623809612769-60436d159e905013ae8715d7.jpeg
paper_authors.fullname: Minho Ryu
paper_authors.type: user
paper_authors.user: bzantium
paper_authors._id: 647eaa82becb41a2729772d0
paper_authors.name: Taekyoon Choi
paper_authors._id: 647eaa82becb41a2729772d1
paper_authors.name: Seungmu Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-07T10:09:40.241Z
paper_authors._id: 61e78b9e68f7ae955e4b0b8e
paper_authors.avatarUrl: /avatars/ecb4c6db9092c0ba84695cf0a33e5a91.svg
paper_authors.fullname: Seungmoo Yang
paper_authors.type: user
paper_authors.user: Moo
paper_authors._id: 647eaa82becb41a2729772d2
paper_authors.name: jiwung Hyun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-06T08:52:26.750Z
paper_authors._id: 61e78aa282b19b93e1a53cbe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662805242568-61e78aa282b19b93e1a53cbe.jpeg
paper_authors.fullname: Jiwung Hyun
paper_authors.type: user
paper_authors.user: kabbi
paper_authors._id: 647eaa82becb41a2729772d3
paper_authors.name: Sungho Park
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-06T08:53:18.455Z
paper_authors._id: 625fc71affe8827cb1d3b1f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660573263107-625fc71affe8827cb1d3b1f8.png
paper_authors.fullname: Sungho Park
paper_authors.type: user
paper_authors.user: naem1023
paper_title: A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean
    Language Models
paper_summary: Polyglot is a pioneering project aimed at enhancing the non-English language
  performance of multilingual language models. Despite the availability of
  various multilingual models such as mBERT (Devlin et al., 2019), XGLM (Lin et
  al., 2022), and BLOOM (Scao et al., 2022), researchers and developers often
  resort to building monolingual models in their respective languages due to the
  dissatisfaction with the current multilingual models non-English language
  capabilities. Addressing this gap, we seek to develop advanced multilingual
  language models that offer improved performance in non-English languages. In
  this paper, we introduce the Polyglot Korean models, which represent a specific
  focus rather than being multilingual in nature. In collaboration with TUNiB,
  our team collected 1.2TB of Korean data meticulously curated for our research
  journey. We made a deliberate decision to prioritize the development of Korean
  models before venturing into multilingual models. This choice was motivated by
  multiple factors: firstly, the Korean models facilitated performance
  comparisons with existing multilingual models; and finally, they catered to the
  specific needs of Korean companies and researchers. This paper presents our
  work in developing the Polyglot Korean models, which propose some steps towards
  addressing the non-English language performance gap in multilingual language
  models.
paper_ai_keywords: multilingual language models mBERT XGLM BLOOM monolingual models non-English language Polyglot Korean models