Upload folder using huggingface_hub

README.md

@@ -10,7 +10,7 @@ repo: kakaocorp/kanana-1.5-2.1b-base
 developers: Kanana LLM
 training_regime: bf16 mixed precision
 ---
-# Kanana
+
 <p align="center">
 <br>
     <picture>
@@ -20,14 +20,14 @@ training_regime: bf16 mixed precision
     </picture>
 </br>
 <p align="center">
-🤗 <a href="https://kko.kakao.com/kananallm">HF Models</a> &nbsp |
-&nbsp 📕 <a href="https://tech.kakao.com/posts/707">Blog Post</a> &nbsp |
+🤗 <a href="https://kko.kakao.com/kananallm">1.5 HF Models</a> &nbsp |
+&nbsp 📕 <a href="https://tech.kakao.com/posts/707">1.5 Blog Post</a> &nbsp |
 &nbsp 📜 <a href="https://arxiv.org/abs/2502.18934">Technical Report</a>
 
 
 <br>
 
-## 🔥 News
+## News 🔥
 
 - ✨`2025/05/23`: Published a [blog post](https://tech.kakao.com/posts/707) about `Kanana 1.5` models and released 🤗[HF model weights](https://kko.kakao.com/kananallm).
 - 📜`2025/02/27`: Released [Technical Report](https://arxiv.org/abs/2502.18934) and 🤗[HF model weights](https://huggingface.co/collections/kakaocorp/kanana-nano-21b-67a326cda1c449c8d4172259).
@@ -41,19 +41,19 @@ training_regime: bf16 mixed precision
 
 - [Kanana 1.5](#kanana-15)
     - [Performance](#performance)
-        - [Kanana 1.5 Base Models](#kanana-15-base-models)
-        - [Kanana 1.5 Instruct Models](#kanana-15-instruct-models)
+        - [Base Model Evaluation](#base-model-evaluation)
+        - [Instruct Model Evaluation](#instruct-model-evaluation)
         - [Long Context](#long-context)
     - [Processing 32K+ Length](#processing-32k-length)
+    - [Contributors](#contributors)
 - [Kanana 1.0](#kanana-10)
-- [License](#license)
 - [Citation](#citation)
-- [Contributors](#contributors)
 - [Contact](#contact)
 
 <br>
 
-## Kanana 1.5
+# Kanana 1.5
+
 `Kanana 1.5`, a newly introduced version of the Kanana model family, presents substantial enhancements in **coding, mathematics, and function calling capabilities** over the previous version, enabling broader application to more complex real-world problems. This new version now can handle __up to 32K tokens length natively and up to 128K tokens using YaRN__, allowing the model to maintain coherence when handling extensive documents or engaging in extended conversations. Furthermore, Kanana 1.5 delivers more natural and accurate conversations through a __refined post-training process__.
 
 <p align="center">
@@ -64,11 +64,9 @@ training_regime: bf16 mixed precision
 > [!Note]
 > Neither the pre-training nor the post-training data includes Kakao user data.
 
-<br>
-
-### Performance
+## Performance
 
-#### Kanana 1.5 Base Models
+### Base Model Evaluation
 <table>
     <tr>
         <th>Models</th>
@@ -162,8 +160,9 @@ training_regime: bf16 mixed precision
     </tr>
 </table>
 
+<br>
 
-#### Kanana 1.5 Instruct Models
+### Instruct Model Evaluation
 <table>
     <tr>
         <th>Models</th>
@@ -299,8 +298,10 @@ training_regime: bf16 mixed precision
 
 > \* Models released under the Apache 2.0 license have been trained on more recent data compared to other models.
 
-#### Long Context
-##### Kanana-1.5-32.5B-Base
+<br>
+
+### Long Context
+#### Kanana-1.5-32.5B-Base
 Below is a Needle-in-a-Haystack performance of `Kanana-1.5-32.5B-Base` model which was trained on a target context length of 32K.
 - (left): evaluated with native 32K context length
 - (right): extended to 128K context length using YaRN
@@ -310,7 +311,7 @@ Below is a Needle-in-a-Haystack performance of `Kanana-1.5-32.5B-Base` model whi
     <img src="assets/performance/niah-32.5b-base.png", width="1000" style="margin: 40px auto;">
 </picture>
 
-##### Kanana-1.5-32.5B-Instruct
+#### Kanana-1.5-32.5B-Instruct
 Below is a Needle-in-a-Haystack performance of `Kanana-1.5-32.5B-Instruct` model which was trained on a target context length of 32K.
 - (left): evaluated with native 32K context length
 - (right): extended to 128K context length using YaRN
@@ -322,7 +323,7 @@ Below is a Needle-in-a-Haystack performance of `Kanana-1.5-32.5B-Instruct` model
 
 <br>
 
-#### Processing 32K+ Length
+## Processing 32K+ Length
 Currently, the `config.json` uploaded to HuggingFace is configured for token lengths of 32,768 or less. To process tokens beyond this length, YaRN must be applied. By updating the `config.json` with the following parameters, you can apply YaRN to handle token sequences up to 128K in length:
 ```json
 "rope_scaling": {
@@ -336,13 +337,19 @@ Currently, the `config.json` uploaded to HuggingFace is configured for token len
 
 <br>
 
+## Contributors
+- Language Model Training: Yunju Bak, Doohae Jung, Boseop Kim, Nayeon Kim, Hojin Lee, Jaesun Park, Minho Ryu
+- Language Model Alignment: Jiyeon Ham, Seungjae Jung, Hyunho Kim, Hyunwoong Ko, Changmin Lee, Daniel Wontae Nam
+- AI Engineering: Youmin Kim, Hyeongju Kim
 
-## Kanana 1.0
-
-### Kanana 1.0 Introduction
+<br>
 
+# Kanana 1.0
 <details>
 <summary>View the details about Kanana 1.0</summary>
+
+<br>
+
 We introduce Kanana, a series of bilingual language models (developed by [Kakao](https://github.com/kakao)) that demonstrate exceeding performance in Korean and competitive performance in English. The computational cost of Kanana is significantly lower than that of state-of-the-art models of similar size. The report details the techniques employed during pre-training to achieve compute-efficient yet competitive models, including high-quality data filtering, staged pre-training, depth up-scaling, and pruning and distillation. Furthermore, the report outlines the methodologies utilized during the post-training of the Kanana models, encompassing supervised fine-tuning and preference optimization, aimed at enhancing their capability for seamless interaction with users. Lastly, the report elaborates on plausible approaches used for language model adaptation to specific scenarios, such as embedding, function calling, and Retrieval Augmented Generation (RAG). The Kanana model series spans from 2.1B to 32.5B parameters with 2.1B models (base, instruct, embedding, function call, and RAG) publicly released to promote research on Korean language models.
 
 > Neither the pre-training nor the post-training data includes Kakao user data.
@@ -352,16 +359,11 @@ We introduce Kanana, a series of bilingual language models (developed by [Kakao]
     <img src="assets/performance/flops-vs-mmlu.jpg", width="700" style="margin: 40px auto;">
 </picture>
 
-</details>
-
-
-### Kanana 1.0 Models Performance
-<details>
-<summary>View detailed performance of Kanana 1.0 models</summary>
+## Performance
 
 Below are partial report on the performance of the `Kanana` model series. Please refer to the [Technical Report](https://arxiv.org/abs/2502.18934) for the full results.
 
-#### Pre-trained Model Performance
+### Base Model Evaluation
 
 <table>
     <tr>
@@ -552,9 +554,9 @@ Below are partial report on the performance of the `Kanana` model series. Please
 <br>
 
 
-#### Post-trained Model Performance
+### Instruct Model Evaluation
 
-##### Instruction-following Benchmarks
+#### Instruction-following Benchmarks
 <table>
     <tr>
         <th>Models</th>
@@ -724,7 +726,7 @@ Below are partial report on the performance of the `Kanana` model series. Please
 
 <br>
 
-##### General Benchmarks
+#### General Benchmarks
 
 <table>
     <tr>
@@ -933,7 +935,7 @@ Below are partial report on the performance of the `Kanana` model series. Please
 
 <br>
 
-#### Embedding Model Performance
+### Embedding Model Performance
 <table>
     <tr>
         <td align="center">Backbone</td>
@@ -969,13 +971,15 @@ Below are partial report on the performance of the `Kanana` model series. Please
     </tr>
 </table>
 
-</details>
+<br>
 
+## Contributors
 
-<br>
+- Pre-training: Yunju Bak, Doohae Jung, Boseop Kim, Nayeon Kim, Hojin Lee, Jaesun Park, Minho Ryu
+- Post-training: Jiyeon Ham, Seungjae Jung, Hyunho Kim, Hyunwoong Ko, Changmin Lee, Daniel Wontae Nam, Kyoung-Woon On
+- Adaptation: Seulye Baeg, Junrae Cho, Taegyeong Eo, Sunghee Jung, Jieun Kang, EungGyun Kim, Eunhwa Kim, Byeongil Ko, Daniel Lee, Donghun Lee, Minchul Lee, Miok Lee, Shinbok Lee, Minho Ryu, Gaeun Seo
 
-## License
-The `Kanana 1.5` models are licensed under [apache-2.0](./LICENSE).
+</details>
 
 <br>
  
@@ -995,22 +999,6 @@ The `Kanana 1.5` models are licensed under [apache-2.0](./LICENSE).
 
 <br>
 
-## Contributors
-- Language Model Training: Yunju Bak, Doohae Jung, Boseop Kim, Nayeon Kim, Hojin Lee, Jaesun Park, Minho Ryu
-- Language Model Alignment: Jiyeon Ham, Seungjae Jung, Hyunho Kim, Hyunwoong Ko, Changmin Lee, Daniel Wontae Nam
-- AI Engineering: Youmin Kim, Hyeongju Kim
-
-<details>
-<summary>Contributors for Kanana 1.0</summary>
-
-- Pre-training: Yunju Bak, Doohae Jung, Boseop Kim, Nayeon Kim, Hojin Lee, Jaesun Park, Minho Ryu
-- Post-training: Jiyeon Ham, Seungjae Jung, Hyunho Kim, Hyunwoong Ko, Changmin Lee, Daniel Wontae Nam, Kyoung-Woon On
-- Adaptation: Seulye Baeg, Junrae Cho, Taegyeong Eo, Sunghee Jung, Jieun Kang, EungGyun Kim, Eunhwa Kim, Byeongil Ko, Daniel Lee, Donghun Lee, Minchul Lee, Miok Lee, Shinbok Lee, Minho Ryu, Gaeun Seo
-
-</details>
-
-<br>
-
 ## Contact
 - Kanana LLM Team Technical Support: [email protected]
 - Business & Partnership Contact: [email protected]