Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@
|
|
9 |
|
10 |
**Introduction**
|
11 |
|
12 |
-
Kwai-Coder-DS-V2-Lite-Base is built on Deepseek-v2-Lite-Base, which has a total of 16B parameters and 2.4B activated parameters. It supports both English and Chinese and underwent
|
13 |
|
14 |
**Performance**
|
15 |
|
|
|
9 |
|
10 |
**Introduction**
|
11 |
|
12 |
+
Kwai-Coder-DS-V2-Lite-Base is built on Deepseek-v2-Lite-Base, which has a total of 16B parameters and 2.4B activated parameters. It supports both English and Chinese and underwent continue pretraining on 800B tokens of high-quality code, math, and Chinese-English text data. The training data consists of 70% code data, 20% math data, and 10% text data (including a large amount of code-related text data). Ultimately, the base model achieved SOTA levels in multiple benchmarks.
|
13 |
|
14 |
**Performance**
|
15 |
|