title: Trillion 7B Technical Report
summary: We introduce Trillion-7B, the most token-efficient Korean-centric
  multilingual LLM available. Our novel Cross-lingual Document Attention (XLDA)
  mechanism enables highly efficient and effective knowledge transfer from
  English to target languages like Korean and Japanese. Combined with optimized
  data mixtures, language-specific filtering, and tailored tokenizer
  construction, Trillion-7B achieves competitive performance while dedicating
  only 10\% of its 2T training tokens to multilingual data and requiring just
  59.4K H100 GPU hours (\$148K) for full training. Comprehensive evaluations
  across 27 benchmarks in four languages demonstrate Trillion-7B's robust
  multilingual performance and exceptional cross-lingual consistency.
paper_authors._id: 680879ead6dc8bf64565c975
paper_authors.name: Sungjun Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:23:37.021Z
paper_authors._id: 67aaee60a8192c1ba3d7d42b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67aaee60a8192c1ba3d7d42b/2pBbWeqUEUyB82EPJpNX4.jpeg
paper_authors.fullname: Sungjun Han
paper_authors.type: user
paper_authors.user: sungjunhan-trl
paper_authors._id: 680879ead6dc8bf64565c976
paper_authors.name: Juyoung Suk
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:08:21.257Z
paper_authors._id: 6138cc1306dd10833d2db64b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6138cc1306dd10833d2db64b/IRX4y-8M4YlzR_8jOwkKp.jpeg
paper_authors.fullname: Juyoung Suk
paper_authors.type: user
paper_authors.user: juyoungml
paper_authors._id: 680879ead6dc8bf64565c977
paper_authors.name: Suyeong An
paper_authors._id: 680879ead6dc8bf64565c978
paper_authors.name: Hyungguk Kim
paper_authors._id: 680879ead6dc8bf64565c979
paper_authors.name: Kyuseok Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:24:00.523Z
paper_authors._id: 6729c4baae4312f3ab001a90
paper_authors.avatarUrl: /avatars/5d454f1695a0a8574e730fd4d884243b.svg
paper_authors.fullname: Kyuseok Kim
paper_authors.type: user
paper_authors.user: kyudolski
paper_authors._id: 680879ead6dc8bf64565c97a
paper_authors.name: Wonsuk Yang
paper_authors._id: 680879ead6dc8bf64565c97b
paper_authors.name: Seungtaek Choi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T09:11:03.864Z
paper_authors._id: 6257adfdb98dcaa7e0de7ab4
paper_authors.avatarUrl: /avatars/ddfc2135104895d09cfce0cd6f10e5fb.svg
paper_authors.fullname: Seungtaek Choi
paper_authors.type: user
paper_authors.user: hist0613
paper_authors._id: 680879ead6dc8bf64565c97c
paper_authors.name: Jamin Shin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T12:41:14.275Z
paper_authors._id: 6188cf3293317afcd1c2df7f
paper_authors.avatarUrl: /avatars/95621801d5b3f3c1a681f1ad6cc66c6a.svg
paper_authors.fullname: Jay Shin
paper_authors.type: user
paper_authors.user: jshin49
paper_title: Trillion 7B Technical Report
paper_summary: We introduce Trillion-7B, the most token-efficient Korean-centric
  multilingual LLM available. Our novel Cross-lingual Document Attention (XLDA)
  mechanism enables highly efficient and effective knowledge transfer from
  English to target languages like Korean and Japanese. Combined with optimized
  data mixtures, language-specific filtering, and tailored tokenizer
  construction, Trillion-7B achieves competitive performance while dedicating
  only 10\% of its 2T training tokens to multilingual data and requiring just
  59.4K H100 GPU hours (\$148K) for full training. Comprehensive evaluations
  across 27 benchmarks in four languages demonstrate Trillion-7B's robust
  multilingual performance and exceptional cross-lingual consistency.
paper_ai_summary: Trillion-7B is a highly efficient multilingual LLM leveraging Cross-lingual Document Attention (XLDA) for knowledge transfer and achieving competitive performance with minimal multilingual training data.
paper_ai_keywords: Cross-lingual Document Attention (XLDA)