Update README.md
Browse files
README.md
CHANGED
@@ -33,7 +33,7 @@ Solar-KoEn represents an advanced iteration of the upstage/SOLAR-10.7B-v1.0 mode
|
|
33 |
|
34 |
**Model Architecture:**
|
35 |
|
36 |
-
SOLAR-
|
37 |
|
38 |
| |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
|
39 |
|---|---|---|---|---|---|---|
|
@@ -57,7 +57,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
|
|
57 |
| Model Name | Vocabulary Size | Description |
|
58 |
| --- | --- | --- |
|
59 |
| Original Solar | 32000 | Sentencepiece BPE |
|
60 |
-
| **Expanded SOLAR-
|
61 |
|
62 |
**Tokenizing "안녕하세요, 오늘은 날씨가 좋네요."**
|
63 |
|
@@ -67,7 +67,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
|
|
67 |
| Model | Tokens |
|
68 |
| --- | --- |
|
69 |
| SOLAR-10.7B | `['▁', '안', '<0xEB>', '<0x85>', '<0x95>', '하', '세', '요', ',', '▁', '오', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '날', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '좋', '네', '요', '.']` |
|
70 |
-
| SOLAR-
|
71 |
|
72 |
**Tokenizing "Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!"**
|
73 |
|
@@ -77,7 +77,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
|
|
77 |
| Model | Tokens |
|
78 |
| --- | --- |
|
79 |
| SOLAR-10.7B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
|
80 |
-
| SOLAR-
|
81 |
|
82 |
# LICENSE
|
83 |
|
@@ -128,7 +128,7 @@ Apache 2.0
|
|
128 |
```
|
129 |
@misc {solar_koen_junbum_taekyoon_2024,
|
130 |
author = { {L. Junbum, Taekyoon Choi} },
|
131 |
-
title = {
|
132 |
year = 2024,
|
133 |
url = { https://huggingface.co/beomi/SOLAR-KOEN-10.8B },
|
134 |
publisher = { Hugging Face }
|
|
|
33 |
|
34 |
**Model Architecture:**
|
35 |
|
36 |
+
SOLAR-KOEN-10.8B is an auto-regressive language model that leverages an optimized transformer architecture derived from Llama-2.
|
37 |
|
38 |
| |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
|
39 |
|---|---|---|---|---|---|---|
|
|
|
57 |
| Model Name | Vocabulary Size | Description |
|
58 |
| --- | --- | --- |
|
59 |
| Original Solar | 32000 | Sentencepiece BPE |
|
60 |
+
| **Expanded SOLAR-KOEN-10.8B** | 46336 | Sentencepiece BPE. Added Korean vocab and merges |
|
61 |
|
62 |
**Tokenizing "안녕하세요, 오늘은 날씨가 좋네요."**
|
63 |
|
|
|
67 |
| Model | Tokens |
|
68 |
| --- | --- |
|
69 |
| SOLAR-10.7B | `['▁', '안', '<0xEB>', '<0x85>', '<0x95>', '하', '세', '요', ',', '▁', '오', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '날', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '좋', '네', '요', '.']` |
|
70 |
+
| SOLAR-KOEN-10.8B | `['▁안', '녕', '하세요', ',', '▁오늘', '은', '▁날', '씨가', '▁좋네요', '.']` |
|
71 |
|
72 |
**Tokenizing "Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!"**
|
73 |
|
|
|
77 |
| Model | Tokens |
|
78 |
| --- | --- |
|
79 |
| SOLAR-10.7B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
|
80 |
+
| SOLAR-KOEN-10.8B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
|
81 |
|
82 |
# LICENSE
|
83 |
|
|
|
128 |
```
|
129 |
@misc {solar_koen_junbum_taekyoon_2024,
|
130 |
author = { {L. Junbum, Taekyoon Choi} },
|
131 |
+
title = { SOLAR-KOEN-10.8B },
|
132 |
year = 2024,
|
133 |
url = { https://huggingface.co/beomi/SOLAR-KOEN-10.8B },
|
134 |
publisher = { Hugging Face }
|