Hugging Face's logo

unsloth
/
Kimi-K2-Base-BF16

Text Generation
Transformers
Safetensors
deepseek_v3
unsloth
conversational
custom_code
text-generation-inference
Model card Files
xet
Community
danielhanchen commited on
Commit
988187a
·
verified ·
1 Parent(s): 1282716

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. LICENSE +27 -0
  3. README.md +841 -0
  4. THIRD_PARTY_NOTICES.md +47 -0
  5. chat_template.jinja +37 -0
  6. config.json +72 -0
  7. configuration_deepseek.py +212 -0
  8. generation_config.json +4 -0
  9. kimi-logo.png +0 -0
  10. model-1-of-61.safetensors +3 -0
  11. model-10-of-61.safetensors +3 -0
  12. model-11-of-61.safetensors +3 -0
  13. model-12-of-61.safetensors +3 -0
  14. model-13-of-61.safetensors +3 -0
  15. model-14-of-61.safetensors +3 -0
  16. model-15-of-61.safetensors +3 -0
  17. model-16-of-61.safetensors +3 -0
  18. model-17-of-61.safetensors +3 -0
  19. model-18-of-61.safetensors +3 -0
  20. model-19-of-61.safetensors +3 -0
  21. model-2-of-61.safetensors +3 -0
  22. model-20-of-61.safetensors +3 -0
  23. model-21-of-61.safetensors +3 -0
  24. model-22-of-61.safetensors +3 -0
  25. model-23-of-61.safetensors +3 -0
  26. model-24-of-61.safetensors +3 -0
  27. model-25-of-61.safetensors +3 -0
  28. model-26-of-61.safetensors +3 -0
  29. model-27-of-61.safetensors +3 -0
  30. model-28-of-61.safetensors +3 -0
  31. model-29-of-61.safetensors +3 -0
  32. model-3-of-61.safetensors +3 -0
  33. model-30-of-61.safetensors +3 -0
  34. model-31-of-61.safetensors +3 -0
  35. model-32-of-61.safetensors +3 -0
  36. model-33-of-61.safetensors +3 -0
  37. model-34-of-61.safetensors +3 -0
  38. model-35-of-61.safetensors +3 -0
  39. model-36-of-61.safetensors +3 -0
  40. model-37-of-61.safetensors +3 -0
  41. model-38-of-61.safetensors +3 -0
  42. model-39-of-61.safetensors +3 -0
  43. model-4-of-61.safetensors +3 -0
  44. model-40-of-61.safetensors +3 -0
  45. model-41-of-61.safetensors +3 -0
  46. model-42-of-61.safetensors +3 -0
  47. model-43-of-61.safetensors +3 -0
  48. model-44-of-61.safetensors +3 -0
  49. model-45-of-61.safetensors +3 -0
  50. model-46-of-61.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
37
+ banner.png filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Modified MIT License
2
+
3
+ Copyright (c) 2025 Moonshot AI
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the “Software”), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ Our only modification part is that, if the Software (or any derivative works
24
+ thereof) is used for any of your commercial products or services that have
25
+ more than 100 million monthly active users, or more than 20 million US dollars
26
+ (or equivalent in other currencies) in monthly revenue, you shall prominently
27
+ display "Kimi K2" on the user interface of such product or service.
README.md ADDED
@@ -0,0 +1,841 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - unsloth
4
+ - unsloth
5
+ base_model:
6
+ - moonshotai/Kimi-K2-Base-BF16
7
+ license: other
8
+ license_name: modified-mit
9
+ library_name: transformers
10
+ ---
11
+ > [!NOTE]
12
+ > Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
13
+ >
14
+
15
+ <div>
16
+ <p style="margin-top: 0;margin-bottom: 0;">
17
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
18
+ </p>
19
+ <div style="display: flex; gap: 5px; align-items: center; ">
20
+ <a href="https://github.com/unslothai/unsloth/">
21
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
22
+ </a>
23
+ <a href="https://discord.gg/unsloth">
24
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
25
+ </a>
26
+ <a href="https://docs.unsloth.ai/">
27
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
28
+ </a>
29
+ </div>
30
+ </div>
31
+
32
+ > [!NOTE]
33
+ > Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
34
+ >
35
+
36
+ <div>
37
+ <p style="margin-top: 0;margin-bottom: 0;">
38
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
39
+ </p>
40
+ <div style="display: flex; gap: 5px; align-items: center; ">
41
+ <a href="https://github.com/unslothai/unsloth/">
42
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
43
+ </a>
44
+ <a href="https://discord.gg/unsloth">
45
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
46
+ </a>
47
+ <a href="https://docs.unsloth.ai/">
48
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
49
+ </a>
50
+ </div>
51
+ </div>
52
+
53
+ <div align="center">
54
+ <picture>
55
+ <img src="figures/kimi-logo.png" width="30%" alt="Kimi K2: Open Agentic Intellignece">
56
+ </picture>
57
+ </div>
58
+
59
+ <hr>
60
+
61
+ <div align="center" style="line-height:1">
62
+ <a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K2-ff6b6b?color=1783ff&logoColor=white"/></a>
63
+ <a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
64
+ </div>
65
+
66
+ <div align="center" style="line-height: 1;">
67
+ <a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
68
+ <a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
69
+ <a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
70
+ </div>
71
+
72
+ <div align="center" style="line-height: 1;">
73
+ <a href="https://github.com/moonshotai/Kimi-K2/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>
74
+ </div>
75
+
76
+ <p align="center">
77
+ <b>📰&nbsp;&nbsp;<a href="https://moonshotai.github.io/Kimi-K2/">Tech Blog</a></b> &nbsp;&nbsp;&nbsp; | &nbsp;&nbsp;&nbsp; <b>📄&nbsp;&nbsp;Paper Link (comming soon)</b>
78
+ </p>
79
+
80
+ ## 1. Model Introduction
81
+
82
+ Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model with 32 billion activated parameters and 1 trillion total parameters. Trained with the Muon optimizer, Kimi K2 achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being meticulously optimized for agentic capabilities.
83
+
84
+ ### Key Features
85
+ - Large-Scale Training: Pre-trained a 1T parameter MoE model on 15.5T tokens with zero training instability.
86
+ - MuonClip Optimizer: We apply the Muon optimizer to an unprecedented scale, and develop novel optimization techniques to resolve instabilities while scaling up.
87
+ - Agentic Intelligence: Specifically designed for tool use, reasoning, and autonomous problem-solving.
88
+
89
+ ### Model Variants
90
+ - **Kimi-K2-Base**: The foundation model, a strong start for researchers and builders who want full control for fine-tuning and custom solutions.
91
+ - **Kimi-K2-Instruct**: The post-trained model best for drop-in, general-purpose chat and agentic experiences. It is a reflex-grade model without long thinking.
92
+
93
+ <div align="center">
94
+ <picture>
95
+ <img src="figures/banner.png" width="80%" alt="Evaluation Results">
96
+ </picture>
97
+ </div>
98
+
99
+ ## 2. Model Summary
100
+
101
+ <div align="center">
102
+
103
+
104
+ | | |
105
+ |:---:|:---:|
106
+ | **Architecture** | Mixture-of-Experts (MoE) |
107
+ | **Total Parameters** | 1T |
108
+ | **Activated Parameters** | 32B |
109
+ | **Number of Layers** (Dense layer included) | 61 |
110
+ | **Number of Dense Layers** | 1 |
111
+ | **Attention Hidden Dimension** | 7168 |
112
+ | **MoE Hidden Dimension** (per Expert) | 2048 |
113
+ | **Number of Attention Heads** | 64 |
114
+ | **Number of Experts** | 384 |
115
+ | **Selected Experts per Token** | 8 |
116
+ | **Number of Shared Experts** | 1 |
117
+ | **Vocabulary Size** | 160K |
118
+ | **Context Length** | 128K |
119
+ | **Attention Mechanism** | MLA |
120
+ | **Activation Function** | SwiGLU |
121
+ </div>
122
+
123
+ ## 3. Evaluation Results
124
+
125
+ #### Instruction model evaluation results
126
+
127
+ <div align="center">
128
+ <table>
129
+ <thead>
130
+ <tr>
131
+ <th align="center">Benchmark</th>
132
+ <th align="center">Metric</th>
133
+ <th align="center"><sup>Kimi K2 Instruct</sup></th>
134
+ <th align="center"><sup>DeepSeek-V3-0324</sup></th>
135
+ <th align="center"><sup>Qwen3-235B-A22B <br><sup>(non-thinking)</sup></sup></th>
136
+ <th align="center"><sup>Claude Sonnet 4 <br><sup>(w/o extended thinking)</sup></sup></th>
137
+ <th align="center"><sup>Claude Opus 4 <br><sup>(w/o extended thinking)</sup></sup></th>
138
+ <th align="center"><sup>GPT-4.1</sup></th>
139
+ <th align="center"><sup>Gemini 2.5 Flash <br> Preview (05-20)</sup></th>
140
+ </tr>
141
+ </thead>
142
+ <tbody>
143
+ <tr>
144
+ <td align="center" colspan=9><strong>Coding Tasks</strong></td>
145
+ </tr>
146
+ <tr>
147
+ <td align="center">LiveCodeBench v6<br><sup>(Aug 24 - May 25)</sup></td>
148
+ <td align="center">Pass@1</td>
149
+ <td align="center"><strong>53.7</strong></td>
150
+ <td align="center">46.9</td>
151
+ <td align="center">37.0</td>
152
+ <td align="center">48.5</td>
153
+ <td align="center">47.4</td>
154
+ <td align="center">44.7</td>
155
+ <td align="center">44.7</td>
156
+ </tr>
157
+ <tr>
158
+ <td align="center">OJBench</td>
159
+ <td align="center">Pass@1</td>
160
+ <td align="center"><strong>27.1</strong></td>
161
+ <td align="center">24.0</td>
162
+ <td align="center">11.3</td>
163
+ <td align="center">15.3</td>
164
+ <td align="center">19.6</td>
165
+ <td align="center">19.5</td>
166
+ <td align="center">19.5</td>
167
+ </tr>
168
+
169
+ <tr>
170
+ <td align="center">MultiPL-E</td>
171
+ <td align="center">Pass@1</td>
172
+ <td align="center"><ins><strong>85.7</strong></ins></td>
173
+ <td align="center">83.1</td>
174
+ <td align="center">78.2</td>
175
+ <td align="center">88.6</td>
176
+ <td align="center"><strong>89.6</strong></td>
177
+ <td align="center">86.7</td>
178
+ <td align="center">85.6</td>
179
+ </tr>
180
+
181
+ <tr>
182
+ <td align="center">SWE-bench Verified <br/><sup>(Agentless Coding)</sup></td>
183
+ <td align="center">Single Patch w/o Test (Acc)</td>
184
+ <td align="center"><ins><strong>51.8</strong></ins></td>
185
+ <td align="center">36.6</td>
186
+ <td align="center">39.4</td>
187
+ <td align="center">50.2</td>
188
+ <td align="center"><strong>53.0</strong></td>
189
+ <td align="center">40.8</td>
190
+ <td align="center">32.6</td>
191
+ </tr>
192
+
193
+ <tr>
194
+ <td align="center" rowspan="2">SWE-bench Verified <br/> <sup>(Agentic Coding)</sup></td>
195
+ <td align="center">Single Attempt (Acc)</td>
196
+ <td align="center"><ins><strong>65.8</strong></ins></td>
197
+ <td align="center">38.8</td>
198
+ <td align="center">34.4</td>
199
+ <td align="center"><strong>72.7</strong><sup>*</sup></td>
200
+ <td align="center">72.5<sup>*</sup></td>
201
+ <td align="center">54.6</td>
202
+ <td align="center">—</td>
203
+ </tr>
204
+
205
+ <tr>
206
+ <!--<td align="center">(Agentic Coding)</td>-->
207
+ <td align="center">Multiple Attempts (Acc)</td>
208
+ <td align="center"><ins><strong>71.6</strong></ins></td>
209
+ <td align="center">—</td>
210
+ <td align="center">—</td>
211
+ <td align="center"><strong>80.2</strong></td>
212
+ <td align="center">79.4<sup>*</sup></td>
213
+ <td align="center">—</td>
214
+ <td align="center">—</td>
215
+ </tr>
216
+
217
+ <tr>
218
+ <td align="center">SWE-bench Multilingual<br /> <sup>(Agentic Coding)</sup></td>
219
+ <td align="center">Single Attempt (Acc)</td>
220
+ <td align="center"><ins><strong>47.3</strong> </ins></td>
221
+ <td align="center">25.8</td>
222
+ <td align="center">20.9</td>
223
+ <td align="center"><strong>51.0</strong></td>
224
+ <td align="center">—</td>
225
+ <td align="center">31.5</td>
226
+ <td align="center">—</td>
227
+ </tr>
228
+
229
+ <tr>
230
+ <td align="center" rowspan="2">TerminalBench</td>
231
+ <td align="center">Inhouse Framework (Acc)</td>
232
+ <td align="center"><ins><strong>30.0</strong></ins></td>
233
+ <td align="center">—</td>
234
+ <td align="center">—</td>
235
+ <td align="center">35.5</td>
236
+ <td align="center"><strong>43.2</strong></td>
237
+ <td align="center">8.3</td>
238
+ <td align="center">—</td>
239
+ </tr>
240
+
241
+ <tr>
242
+ <!--<td align="center">TerminalBench</td>-->
243
+ <td align="center">Terminus (Acc)</td>
244
+ <td align="center"><ins><strong>25.0</strong> </ins></td>
245
+ <td align="center">16.3</td>
246
+ <td align="center">6.6</td>
247
+ <td align="center">—</td>
248
+ <td align="center">—</td>
249
+ <td align="center"><strong>30.3</strong></td>
250
+ <td align="center">16.8</td>
251
+ </tr>
252
+ <tr>
253
+ <td align="center">Aider-Polyglot</td>
254
+ <td align="center">Acc</td>
255
+ <td align="center">60.0</td>
256
+ <td align="center">55.1</td>
257
+ <td align="center"><ins><strong>61.8</strong></ins></td>
258
+ <td align="center">56.4</td>
259
+ <td align="center"><strong>70.7</strong></td>
260
+ <td align="center">52.4</td>
261
+ <td align="center">44.0</td>
262
+ </tr>
263
+ <tr>
264
+ <td align="center" colspan=9><strong>Tool Use Tasks</strong></td>
265
+ </tr>
266
+ <tr>
267
+ <td align="center">Tau2 retail</td>
268
+ <td align="center">Avg@4</td>
269
+ <td align="center"><ins><strong>70.6</strong></ins></td>
270
+ <td align="center">69.1</td>
271
+ <td align="center">57.0</td>
272
+ <td align="center">75.0</td>
273
+ <td align="center"><strong>81.8</strong></td>
274
+ <td align="center">74.8</td>
275
+ <td align="center">64.3</td>
276
+ </tr>
277
+ <tr>
278
+ <td align="center">Tau2 airline</td>
279
+ <td align="center">Avg@4</td>
280
+ <td align="center"><ins><strong>56.5</strong></ins></td>
281
+ <td align="center">39.0</td>
282
+ <td align="center">26.5</td>
283
+ <td align="center">55.5</td>
284
+ <td align="center"><strong>60.0</strong></td>
285
+ <td align="center">54.5</td>
286
+ <td align="center">42.5</td>
287
+ </tr>
288
+ <tr>
289
+ <td align="center">Tau2 telecom</td>
290
+ <td align="center">Avg@4</td>
291
+ <td align="center"><strong>65.8</strong></td>
292
+ <td align="center">32.5</td>
293
+ <td align="center">22.1</td>
294
+ <td align="center">45.2</td>
295
+ <td align="center">57.0</td>
296
+ <td align="center">38.6</td>
297
+ <td align="center">16.9</td>
298
+ </tr>
299
+ <tr>
300
+ <td align="center">AceBench</td>
301
+ <td align="center">Acc</td>
302
+ <td align="center"><ins><strong>76.5</strong></ins></td>
303
+ <td align="center">72.7</td>
304
+ <td align="center">70.5</td>
305
+ <td align="center">76.2</td>
306
+ <td align="center">75.6</td>
307
+ <td align="center"><strong>80.1</strong></td>
308
+ <td align="center">74.5</td>
309
+ </tr>
310
+ <tr>
311
+ <td align="center" colspan=9><strong>Math &amp; STEM Tasks</strong></td>
312
+ </tr>
313
+ <tr>
314
+ <td align="center">AIME 2024</td>
315
+ <td align="center">Avg@64</td>
316
+ <td align="center"><strong>69.6</strong></td>
317
+ <td align="center">59.4<sup>*</sup></td>
318
+ <td align="center">40.1<sup>*</sup></td>
319
+ <td align="center">43.4</td>
320
+ <td align="center">48.2</td>
321
+ <td align="center">46.5</td>
322
+ <td align="center">61.3</td>
323
+ </tr>
324
+ <tr>
325
+ <td align="center">AIME 2025</td>
326
+ <td align="center">Avg@64</td>
327
+ <td align="center"><strong>49.5</strong></td>
328
+ <td align="center">46.7</td>
329
+ <td align="center">24.7<sup>*</sup></td>
330
+ <td align="center">33.1<sup>*</sup></td>
331
+ <td align="center">33.9<sup>*</sup></td>
332
+ <td align="center">37.0</td>
333
+ <td align="center">46.6</td>
334
+ </tr>
335
+ <tr>
336
+ <td align="center">MATH-500</td>
337
+ <td align="center">Acc</td>
338
+ <td align="center"><strong>97.4</strong></td>
339
+ <td align="center">94.0<sup>*</sup></td>
340
+ <td align="center">91.2<sup>*</sup></td>
341
+ <td align="center">94.0</td>
342
+ <td align="center">94.4</td>
343
+ <td align="center">92.4</td>
344
+ <td align="center">95.4</td>
345
+ </tr>
346
+ <tr>
347
+ <td align="center">HMMT 2025</td>
348
+ <td align="center">Avg@32</td>
349
+ <td align="center"><strong>38.8</strong></td>
350
+ <td align="center">27.5</td>
351
+ <td align="center">11.9</td>
352
+ <td align="center">15.9</td>
353
+ <td align="center">15.9</td>
354
+ <td align="center">19.4</td>
355
+ <td align="center">34.7</td>
356
+ </tr>
357
+ <tr>
358
+ <td align="center">CNMO 2024</td>
359
+ <td align="center">Avg@16</td>
360
+ <td align="center">74.3</td>
361
+ <td align="center"><ins><strong>74.7</strong></ins></td>
362
+ <td align="center">48.6</td>
363
+ <td align="center">60.4</td>
364
+ <td align="center">57.6</td>
365
+ <td align="center">56.6</td>
366
+ <td align="center"><strong>75.0</strong></td>
367
+ </tr>
368
+ <tr>
369
+ <td align="center">PolyMath-en</td>
370
+ <td align="center">Avg@4</td>
371
+ <td align="center"><strong>65.1</strong></td>
372
+ <td align="center">59.5</td>
373
+ <td align="center">51.9</td>
374
+ <td align="center">52.8</td>
375
+ <td align="center">49.8</td>
376
+ <td align="center">54.0</td>
377
+ <td align="center">49.9</td>
378
+ </tr>
379
+
380
+ <tr>
381
+ <td align="center">ZebraLogic</td>
382
+ <td align="center">Acc</td>
383
+ <td align="center"><strong>89.0</strong></td>
384
+ <td align="center">84.0</td>
385
+ <td align="center">37.7<sup>*</sup></td>
386
+ <td align="center">73.7</td>
387
+ <td align="center">59.3</td>
388
+ <td align="center">58.5</td>
389
+ <td align="center">57.9</td>
390
+ </tr>
391
+
392
+ <tr>
393
+ <td align="center">AutoLogi</td>
394
+ <td align="center">Acc</td>
395
+ <td align="center"><ins><strong>89.5</strong></ins></td>
396
+ <td align="center">88.9</td>
397
+ <td align="center">83.3</td>
398
+ <td align="center"><strong>89.8</strong></td>
399
+ <td align="center">86.1</td>
400
+ <td align="center">88.2</td>
401
+ <td align="center">84.1</td>
402
+ </tr>
403
+
404
+ <tr>
405
+ <td align="center">GPQA-Diamond</td>
406
+ <td align="center">Avg@8</td>
407
+ <td align="center"><strong>75.1</strong></td>
408
+ <td align="center">68.4<sup>*</sup></td>
409
+ <td align="center">62.9<sup>*</sup></td>
410
+ <td align="center">70.0<sup>*</sup></td>
411
+ <td align="center">74.9<sup>*</sup></td>
412
+ <td align="center">66.3</td>
413
+ <td align="center">68.2</td>
414
+ </tr>
415
+
416
+ <tr>
417
+ <td align="center">SuperGPQA</td>
418
+ <td align="center">Acc</td>
419
+ <td align="center"><strong>57.2</strong></td>
420
+ <td align="center">53.7</td>
421
+ <td align="center">50.2</td>
422
+ <td align="center">55.7</td>
423
+ <td align="center">56.5</td>
424
+ <td align="center">50.8</td>
425
+ <td align="center">49.6</td>
426
+ </tr>
427
+
428
+ <tr>
429
+ <td align="center">Humanity's Last Exam<br><sup>(Text Only)</sup></td>
430
+ <td align="center">-</td>
431
+ <td align="center">4.7</td>
432
+ <td align="center">5.2</td>
433
+ <td align="center"><ins><strong>5.7</strong></ins></td>
434
+ <td align="center">5.8</td>
435
+ <td align="center"><strong>7.1</strong></td>
436
+ <td align="center">3.7</td>
437
+ <td align="center">5.6</td>
438
+ </tr>
439
+
440
+ <tr>
441
+ <td align="center" colspan=9><strong>General Tasks</strong></td>
442
+ </tr>
443
+
444
+ <tr>
445
+ <td align="center">MMLU</td>
446
+ <td align="center">EM</td>
447
+ <td align="center"><ins><strong>89.5</strong></ins></td>
448
+ <td align="center">89.4</td>
449
+ <td align="center">87.0</td>
450
+ <td align="center">91.5</td>
451
+ <td align="center"><strong>92.9</strong></td>
452
+ <td align="center">90.4</td>
453
+ <td align="center">90.1</td>
454
+ </tr>
455
+
456
+ <tr>
457
+ <td align="center">MMLU-Redux</td>
458
+ <td align="center">EM</td>
459
+ <td align="center"><ins><strong>92.7</strong></ins></td>
460
+ <td align="center">90.5</td>
461
+ <td align="center">89.2</td>
462
+ <td align="center">93.6</td>
463
+ <td align="center"><strong>94.2</strong></td>
464
+ <td align="center">92.4</td>
465
+ <td align="center">90.6</td>
466
+ </tr>
467
+
468
+ <tr>
469
+ <td align="center">MMLU-Pro</td>
470
+ <td align="center">EM</td>
471
+ <td align="center">81.1</td>
472
+ <td align="center"><ins><strong>81.2</strong></ins><sup>*</sup></td>
473
+ <td align="center">77.3</td>
474
+ <td align="center">83.7</td>
475
+ <td align="center"><strong>86.6</strong></td>
476
+ <td align="center">81.8</td>
477
+ <td align="center">79.4</td>
478
+ </tr>
479
+
480
+ <tr>
481
+ <td align="center">IFEval</td>
482
+ <td align="center">Prompt Strict</td>
483
+ <td align="center"><strong>89.8</strong></td>
484
+ <td align="center">81.1</td>
485
+ <td align="center">83.2<sup>*</sup></td>
486
+ <td align="center">87.6</td>
487
+ <td align="center">87.4</td>
488
+ <td align="center">88.0</td>
489
+ <td align="center">84.3</td>
490
+ </tr>
491
+
492
+ <tr>
493
+ <td align="center">Multi-Challenge</td>
494
+ <td align="center">Acc</td>
495
+ <td align="center"><strong>54.1</strong></td>
496
+ <td align="center">31.4</td>
497
+ <td align="center">34.0</td>
498
+ <td align="center">46.8</td>
499
+ <td align="center">49.0</td>
500
+ <td align="center">36.4</td>
501
+ <td align="center">39.5</td>
502
+ </tr>
503
+
504
+ <tr>
505
+ <td align="center">SimpleQA</td>
506
+ <td align="center">Correct</td>
507
+ <td align="center"><ins><strong>31.0</strong></ins></td>
508
+ <td align="center">27.7</td>
509
+ <td align="center">13.2</td>
510
+ <td align="center">15.9</td>
511
+ <td align="center">22.8</td>
512
+ <td align="center"><strong>42.3</strong></td>
513
+ <td align="center">23.3</td>
514
+ </tr>
515
+
516
+ <tr>
517
+ <td align="center">Livebench</td>
518
+ <td align="center">Pass@1</td>
519
+ <td align="center"><strong>76.4</strong></td>
520
+ <td align="center">72.4</td>
521
+ <td align="center">67.6</td>
522
+ <td align="center">74.8</td>
523
+ <td align="center">74.6</td>
524
+ <td align="center">69.8</td>
525
+ <td align="center">67.8</td>
526
+ </tr>
527
+ </tbody>
528
+ </table>
529
+ </div>
530
+ <sup>
531
+ • Bold denotes global SOTA, and underlined denotes open-source SOTA.
532
+ </sup><br/><sup>
533
+ • Data points marked with * are taken directly from the model's tech report or blog.
534
+ </sup><br/><sup>
535
+ • All metrics, except for SWE-bench Verified (Agentless), are evaluated with an 8k output token length. SWE-bench Verified (Agentless) is limited to a 16k output token length.
536
+ </sup><br/><sup>
537
+ • Kimi K2 achieves 65.8% pass@1 on the SWE-bench Verified tests with bash/editor tools (single-attempt patches, no test-time compute). It also achieves a 47.3% pass@1 on the SWE-bench Multilingual tests under the same conditions. Additionally, we report results on SWE-bench Verified tests (71.6%) that leverage parallel test-time compute by sampling multiple sequences and selecting the single best via an internal scoring model.
538
+ </sup><br/><sup>
539
+ • To ensure the stability of the evaluation, we employed avg@k on the AIME, HMMT, CNMO, PolyMath-en, GPQA-Diamond, EvalPlus, Tau2.
540
+ </sup><br/><sup>
541
+ • Some data points have been omitted due to prohibitively expensive evaluation costs.
542
+ </sup>
543
+
544
+ ---
545
+
546
+ #### Base model evaluation results
547
+
548
+ <div align="center">
549
+
550
+ <table>
551
+ <thead>
552
+ <tr>
553
+ <th align="center">Benchmark</th>
554
+ <th align="center">Metric</th>
555
+ <th align="center">Shot</th>
556
+ <th align="center">Kimi K2 Base</th>
557
+ <th align="center">Deepseek-V3-Base</th>
558
+ <th align="center">Qwen2.5-72B</th>
559
+ <th align="center">Llama 4 Maverick</th>
560
+ </tr>
561
+ </thead>
562
+ <tbody>
563
+ <tr>
564
+ <td align="center" colspan="7"><strong>General Tasks</strong></td>
565
+ </tr>
566
+ <tr>
567
+ <td align="center">MMLU</td>
568
+ <td align="center">EM</td>
569
+ <td align="center">5-shot</td>
570
+ <td align="center"><strong>87.8</strong></td>
571
+ <td align="center">87.1</td>
572
+ <td align="center">86.1</td>
573
+ <td align="center">84.9</td>
574
+ </tr>
575
+ <tr>
576
+ <td align="center">MMLU-pro</td>
577
+ <td align="center">EM</td>
578
+ <td align="center">5-shot</td>
579
+ <td align="center"><strong>69.2</strong></td>
580
+ <td align="center">60.6</td>
581
+ <td align="center">62.8</td>
582
+ <td align="center">63.5</td>
583
+ </tr>
584
+ <tr>
585
+ <td align="center">MMLU-redux-2.0</td>
586
+ <td align="center">EM</td>
587
+ <td align="center">5-shot</td>
588
+ <td align="center"><strong>90.2</strong></td>
589
+ <td align="center">89.5</td>
590
+ <td align="center">87.8</td>
591
+ <td align="center">88.2</td>
592
+ </tr>
593
+ <tr>
594
+ <td align="center">SimpleQA</td>
595
+ <td align="center">Correct</td>
596
+ <td align="center">5-shot</td>
597
+ <td align="center"><strong>35.3</strong></td>
598
+ <td align="center">26.5</td>
599
+ <td align="center">10.3</td>
600
+ <td align="center">23.7</td>
601
+ </tr>
602
+ <tr>
603
+ <td align="center">TriviaQA</td>
604
+ <td align="center">EM</td>
605
+ <td align="center">5-shot</td>
606
+ <td align="center"><strong>85.1</strong></td>
607
+ <td align="center">84.1</td>
608
+ <td align="center">76.0</td>
609
+ <td align="center">79.3</td>
610
+ </tr>
611
+ <tr>
612
+ <td align="center">GPQA-Diamond</td>
613
+ <td align="center">Avg@8</td>
614
+ <td align="center">5-shot</td>
615
+ <td align="center">48.1</td>
616
+ <td align="center"><strong>50.5</strong></td>
617
+ <td align="center">40.8</td>
618
+ <td align="center">49.4</td>
619
+ </tr>
620
+ <tr>
621
+ <td align="center">SuperGPQA</td>
622
+ <td align="center">EM</td>
623
+ <td align="center">5-shot</td>
624
+ <td align="center"><strong>44.7</strong></td>
625
+ <td align="center">39.2</td>
626
+ <td align="center">34.2</td>
627
+ <td align="center">38.8</td>
628
+ </tr>
629
+ <tr>
630
+ <td align="center" colspan="7"><strong>Coding Tasks</strong></td>
631
+ </tr>
632
+ <tr>
633
+ <td align="center">LiveCodeBench v6</td>
634
+ <td align="center">Pass@1</td>
635
+ <td align="center">1-shot</td>
636
+ <td align="center"><strong>26.3</strong></td>
637
+ <td align="center">22.9</td>
638
+ <td align="center">21.1</td>
639
+ <td align="center">25.1</td>
640
+ </tr>
641
+ <tr>
642
+ <td align="center">EvalPlus</td>
643
+ <td align="center">Pass@1</td>
644
+ <td align="center">-</td>
645
+ <td align="center"><strong>80.3</strong></td>
646
+ <td align="center">65.6</td>
647
+ <td align="center">66.0</td>
648
+ <td align="center">65.5</td>
649
+ </tr>
650
+ <tr>
651
+ <td align="center" colspan="7"><strong>Mathematics Tasks</strong></td>
652
+ </tr>
653
+ <tr>
654
+ <td align="center">MATH</td>
655
+ <td align="center">EM</td>
656
+ <td align="center">4-shot</td>
657
+ <td align="center"><strong>70.2</strong></td>
658
+ <td align="center">60.1</td>
659
+ <td align="center">61.0</td>
660
+ <td align="center">63.0</td>
661
+ </tr>
662
+ <tr>
663
+ <td align="center">GSM8k</td>
664
+ <td align="center">EM</td>
665
+ <td align="center">8-shot</td>
666
+ <td align="center"><strong>92.1</strong></td>
667
+ <td align="center">91.7</td>
668
+ <td align="center">90.4</td>
669
+ <td align="center">86.3</td>
670
+ </tr>
671
+ <tr>
672
+ <td align="center" colspan="7"><strong>Chinese Tasks</strong></td>
673
+ </tr>
674
+ <tr>
675
+ <td align="center">C-Eval</td>
676
+ <td align="center">EM</td>
677
+ <td align="center">5-shot</td>
678
+ <td align="center"><strong>92.5</strong></td>
679
+ <td align="center">90.0</td>
680
+ <td align="center">90.9</td>
681
+ <td align="center">80.9</td>
682
+ </tr>
683
+ <tr>
684
+ <td align="center">CSimpleQA</td>
685
+ <td align="center">Correct</td>
686
+ <td align="center">5-shot</td>
687
+ <td align="center"><strong>77.6</strong></td>
688
+ <td align="center">72.1</td>
689
+ <td align="center">50.5</td>
690
+ <td align="center">53.5</td>
691
+ </tr>
692
+ </tbody>
693
+ </table>
694
+ </div>
695
+ <sup>
696
+ • We only evaluate open-source pretrained models in this work. We report results for Qwen2.5-72B because the base checkpoint for Qwen3-235B-A22B was not open-sourced at the time of our study.
697
+ </sup><br/><sup>
698
+ • All models are evaluated using the same evaluation protocol.
699
+
700
+ </sup>
701
+
702
+
703
+ ## 4. Deployment
704
+ > [!Note]
705
+ > You can access Kimi K2's API on https://platform.moonshot.ai , we provide OpenAI/Anthropic-compatible API for you.
706
+ >
707
+ > The Anthropic-compatible API maps temperature by `real_temperature = request_temperature * 0.6` for better compatible with existing applications.
708
+
709
+ Our model checkpoints are stored in the block-fp8 format, you can find it on [Huggingface](https://huggingface.co/moonshotai/Kimi-K2-Instruct).
710
+
711
+ Currently, Kimi-K2 is recommended to run on the following inference engines:
712
+
713
+ * vLLM
714
+ * SGLang
715
+ * KTransformers
716
+ * TensorRT-LLM
717
+
718
+ Deployment examples for vLLM and SGLang can be found in the [Model Deployment Guide](docs/deploy_guidance.md).
719
+
720
+ ---
721
+
722
+ ## 5. Model Usage
723
+
724
+ ### Chat Completion
725
+
726
+ Once the local inference service is up, you can interact with it through the chat endpoint:
727
+
728
+ ```python
729
+ def simple_chat(client: OpenAI, model_name: str):
730
+ messages = [
731
+ {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
732
+ {"role": "user", "content": [{"type": "text", "text": "Please give a brief self-introduction."}]},
733
+ ]
734
+ response = client.chat.completions.create(
735
+ model=model_name,
736
+ messages=messages,
737
+ stream=False,
738
+ temperature=0.6,
739
+ max_tokens=256
740
+ )
741
+ print(response.choices[0].message.content)
742
+ ```
743
+
744
+ > [!NOTE]
745
+ > The recommended temperature for Kimi-K2-Instruct is `temperature = 0.6`.
746
+ > If no special instructions are required, the system prompt above is a good default.
747
+
748
+ ---
749
+
750
+ ### Tool Calling
751
+
752
+ Kimi-K2-Instruct has strong tool-calling capabilities.
753
+ To enable them, you need to pass the list of available tools in each request, then the model will autonomously decide when and how to invoke them.
754
+
755
+ The following example demonstrates calling a weather tool end-to-end:
756
+
757
+ ```python
758
+ # Your tool implementation
759
+ def get_weather(city: str) -> dict:
760
+ return {"weather": "Sunny"}
761
+
762
+ # Tool schema definition
763
+ tools = [{
764
+ "type": "function",
765
+ "function": {
766
+ "name": "get_weather",
767
+ "description": "Retrieve current weather information. Call this when the user asks about the weather.",
768
+ "parameters": {
769
+ "type": "object",
770
+ "required": ["city"],
771
+ "properties": {
772
+ "city": {
773
+ "type": "string",
774
+ "description": "Name of the city"
775
+ }
776
+ }
777
+ }
778
+ }
779
+ }]
780
+
781
+ # Map tool names to their implementations
782
+ tool_map = {
783
+ "get_weather": get_weather
784
+ }
785
+
786
+ def tool_call_with_client(client: OpenAI, model_name: str):
787
+ messages = [
788
+ {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
789
+ {"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
790
+ ]
791
+ finish_reason = None
792
+ while finish_reason is None or finish_reason == "tool_calls":
793
+ completion = client.chat.completions.create(
794
+ model=model_name,
795
+ messages=messages,
796
+ temperature=0.6,
797
+ tools=tools, # tool list defined above
798
+ tool_choice="auto"
799
+ )
800
+ choice = completion.choices[0]
801
+ finish_reason = choice.finish_reason
802
+ if finish_reason == "tool_calls":
803
+ messages.append(choice.message)
804
+ for tool_call in choice.message.tool_calls:
805
+ tool_call_name = tool_call.function.name
806
+ tool_call_arguments = json.loads(tool_call.function.arguments)
807
+ tool_function = tool_map[tool_call_name]
808
+ tool_result = tool_function(**tool_call_arguments)
809
+ print("tool_result:", tool_result)
810
+
811
+ messages.append({
812
+ "role": "tool",
813
+ "tool_call_id": tool_call.id,
814
+ "name": tool_call_name,
815
+ "content": json.dumps(tool_result)
816
+ })
817
+ print("-" * 100)
818
+ print(choice.message.content)
819
+ ```
820
+
821
+ The `tool_call_with_client` function implements the pipeline from user query to tool execution.
822
+ This pipeline requires the inference engine to support Kimi-K2’s native tool-parsing logic.
823
+ For streaming output and manual tool-parsing, see the [Tool Calling Guide](docs/tool_call_guidance.md).
824
+
825
+ ---
826
+
827
+ ## 6. License
828
+
829
+ Both the code repository and the model weights are released under the [Modified MIT License](LICENSE).
830
+
831
+ ---
832
+
833
+ ## 7. Third Party Notices
834
+
835
+ See [THIRD PARTY NOTICES](THIRD_PARTY_NOTICES.md)
836
+
837
+ ---
838
+
839
+ ## 7. Contact Us
840
+
841
+ If you have any questions, please reach out at [[email protected]](mailto:[email protected]).
THIRD_PARTY_NOTICES.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # THIRD_PARTY_NOTICES
2
+
3
+ This file lists third-party software contained in Kimi-K2 along with their licenses, in compliance with the redistribution clauses of those licenses.
4
+
5
+ ---
6
+
7
+ ## 1. DeepSeek-V3
8
+
9
+ Our model archietecture is DeepSeek-V3-like. Some of modeling codes are copied from the source repository.
10
+
11
+ - **Source Repository**
12
+ https://huggingface.co/deepseek-ai/DeepSeek-V3
13
+
14
+ - **Files / Directories Used**
15
+ - configuation_deepseek.py
16
+ - modeling_deepseek.py
17
+
18
+ - **License Type**
19
+ MIT License
20
+
21
+ - **Copyright Notice**
22
+ Copyright (c) 2023 DeepSeek
23
+
24
+ - **Full License Text**
25
+ ```
26
+ MIT License
27
+
28
+ Copyright (c) 2023 DeepSeek
29
+
30
+ Permission is hereby granted, free of charge, to any person obtaining a copy
31
+ of this software and associated documentation files (the "Software"), to deal
32
+ in the Software without restriction, including without limitation the rights
33
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
34
+ copies of the Software, and to permit persons to whom the Software is
35
+ furnished to do so, subject to the following conditions:
36
+
37
+ The above copyright notice and this permission notice shall be included in all
38
+ copies or substantial portions of the Software.
39
+
40
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
41
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
42
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
43
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
44
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
45
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
46
+ SOFTWARE.
47
+ ```
chat_template.jinja ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {% if tools -%}
2
+ {{ '<|im_system|>tool_declare<|im_middle|>' -}}
3
+ {{- tools | tojson -}}
4
+ {{ '<|im_end|>' -}}
5
+ {%- endif -%}
6
+
7
+ {%- for message in messages -%}
8
+ {%- if loop.first and messages[0]['role'] != 'system' -%}
9
+ {{ '<|im_system|>system<|im_middle|>You are a helpful assistant<|im_end|>' }}
10
+ {%- endif -%}
11
+ {%- if message['role'] == 'system' -%}
12
+ {{ '<|im_system|>system<|im_middle|>' }}
13
+ {%- elif message['role'] == 'user' -%}
14
+ {{ '<|im_user|>user<|im_middle|>' }}
15
+ {%- elif message['role'] == 'assistant' -%}
16
+ {{ '<|im_assistant|>assistant<|im_middle|>' }}
17
+ {%- elif message['role'] == 'tool' -%}
18
+ {{ '<|im_system|>tool<|im_middle|>' }}
19
+ {%- endif -%}
20
+
21
+ {%- if message['content'] is string -%}
22
+ {{- message['content'] + '<|im_end|>' -}}
23
+ {%- else -%}
24
+ {%- for content in message['content'] -%}
25
+ {%- if content['type'] == 'image' or 'image' in content or 'image_url' in content -%}
26
+ {{ '<|media_start|>image<|media_content|><|media_pad|><|media_end|>' }}
27
+ {%- else -%}
28
+ {{ content['text'] }}
29
+ {%- endif -%}
30
+ {%- endfor -%}
31
+ {{ '<|im_end|>' }}
32
+ {%- endif -%}
33
+ {%- endfor -%}
34
+
35
+ {%- if add_generation_prompt -%}
36
+ {{ '<|im_assistant|>assistant<|im_middle|>' }}
37
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "DeepseekV3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "auto_map": {
8
+ "AutoConfig": "configuration_deepseek.DeepseekV3Config",
9
+ "AutoModel": "modeling_deepseek.DeepseekV3Model",
10
+ "AutoModelForCausalLM": "modeling_deepseek.DeepseekV3ForCausalLM"
11
+ },
12
+ "aux_loss_alpha": 0.001,
13
+ "bos_token_id": 163584,
14
+ "eos_token_id": 163585,
15
+ "ep_size": 1,
16
+ "first_k_dense_replace": 1,
17
+ "hidden_act": "silu",
18
+ "hidden_size": 7168,
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 18432,
21
+ "kv_lora_rank": 512,
22
+ "max_position_embeddings": 131072,
23
+ "model_type": "deepseek_v3",
24
+ "moe_intermediate_size": 2048,
25
+ "moe_layer_freq": 1,
26
+ "n_group": 1,
27
+ "n_routed_experts": 384,
28
+ "n_shared_experts": 1,
29
+ "norm_topk_prob": true,
30
+ "num_attention_heads": 64,
31
+ "num_experts_per_tok": 8,
32
+ "num_hidden_layers": 61,
33
+ "num_key_value_heads": 64,
34
+ "num_nextn_predict_layers": 0,
35
+ "pad_token_id": 163839,
36
+ "pretraining_tp": 1,
37
+ "q_lora_rank": 1536,
38
+ "qk_nope_head_dim": 128,
39
+ "qk_rope_head_dim": 64,
40
+ "quantization_config": {
41
+ "activation_scheme": "dynamic",
42
+ "fmt": "e4m3",
43
+ "quant_method": "fp8",
44
+ "weight_block_size": [
45
+ 128,
46
+ 128
47
+ ]
48
+ },
49
+ "rms_norm_eps": 1e-06,
50
+ "rope_scaling": {
51
+ "beta_fast": 1.0,
52
+ "beta_slow": 1.0,
53
+ "factor": 32.0,
54
+ "mscale": 1.0,
55
+ "mscale_all_dim": 1.0,
56
+ "original_max_position_embeddings": 4096,
57
+ "type": "yarn"
58
+ },
59
+ "rope_theta": 50000.0,
60
+ "routed_scaling_factor": 2.827,
61
+ "scoring_func": "sigmoid",
62
+ "seq_aux": true,
63
+ "tie_word_embeddings": false,
64
+ "topk_group": 1,
65
+ "topk_method": "noaux_tc",
66
+ "torch_dtype": "bfloat16",
67
+ "transformers_version": "4.53.2",
68
+ "unsloth_fixed": true,
69
+ "use_cache": true,
70
+ "v_head_dim": 128,
71
+ "vocab_size": 163840
72
+ }
configuration_deepseek.py ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copy from https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/configuration_deepseek.py
2
+
3
+ from transformers.configuration_utils import PretrainedConfig
4
+ from transformers.utils import logging
5
+
6
+ logger = logging.get_logger(__name__)
7
+
8
+ DEEPSEEK_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
9
+ class DeepseekV3Config(PretrainedConfig):
10
+ r"""
11
+ This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek
12
+ model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
13
+ defaults will yield a similar configuration to that of the DeepSeek-V3.
14
+
15
+ Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
16
+ documentation from [`PretrainedConfig`] for more information.
17
+
18
+
19
+ Args:
20
+ vocab_size (`int`, *optional*, defaults to 129280):
21
+ Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the
22
+ `inputs_ids` passed when calling [`DeepseekV3Model`]
23
+ hidden_size (`int`, *optional*, defaults to 4096):
24
+ Dimension of the hidden representations.
25
+ intermediate_size (`int`, *optional*, defaults to 11008):
26
+ Dimension of the MLP representations.
27
+ moe_intermediate_size (`int`, *optional*, defaults to 1407):
28
+ Dimension of the MoE representations.
29
+ num_hidden_layers (`int`, *optional*, defaults to 32):
30
+ Number of hidden layers in the Transformer decoder.
31
+ num_nextn_predict_layers (`int`, *optional*, defaults to 1):
32
+ Number of nextn predict layers in the DeepSeekV3 Model.
33
+ num_attention_heads (`int`, *optional*, defaults to 32):
34
+ Number of attention heads for each attention layer in the Transformer decoder.
35
+ n_shared_experts (`int`, *optional*, defaults to None):
36
+ Number of shared experts, None means dense model.
37
+ n_routed_experts (`int`, *optional*, defaults to None):
38
+ Number of routed experts, None means dense model.
39
+ routed_scaling_factor (`float`, *optional*, defaults to 1.0):
40
+ Scaling factor or routed experts.
41
+ topk_method (`str`, *optional*, defaults to `gready`):
42
+ Topk method used in routed gate.
43
+ n_group (`int`, *optional*, defaults to None):
44
+ Number of groups for routed experts.
45
+ topk_group (`int`, *optional*, defaults to None):
46
+ Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups).
47
+ num_experts_per_tok (`int`, *optional*, defaults to None):
48
+ Number of selected experts, None means dense model.
49
+ moe_layer_freq (`int`, *optional*, defaults to 1):
50
+ The frequency of the MoE layer: one expert layer for every `moe_layer_freq - 1` dense layers.
51
+ first_k_dense_replace (`int`, *optional*, defaults to 0):
52
+ Number of dense layers in shallow layers(embed->dense->dense->...->dense->moe->moe...->lm_head).
53
+ \--k dense layers--/
54
+ norm_topk_prob (`bool`, *optional*, defaults to False):
55
+ Whether to normalize the weights of the routed experts.
56
+ scoring_func (`str`, *optional*, defaults to 'softmax'):
57
+ Method of computing expert weights.
58
+ aux_loss_alpha (`float`, *optional*, defaults to 0.001):
59
+ Auxiliary loss weight coefficient.
60
+ seq_aux = (`bool`, *optional*, defaults to True):
61
+ Whether to compute the auxiliary loss for each individual sample.
62
+ num_key_value_heads (`int`, *optional*):
63
+ This is the number of key_value heads that should be used to implement Grouped Query Attention. If
64
+ `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
65
+ `num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When
66
+ converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
67
+ by meanpooling all the original heads within that group. For more details checkout [this
68
+ paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to
69
+ `num_attention_heads`.
70
+ hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
71
+ The non-linear activation function (function or string) in the decoder.
72
+ max_position_embeddings (`int`, *optional*, defaults to 2048):
73
+ The maximum sequence length that this model might ever be used with.
74
+ initializer_range (`float`, *optional*, defaults to 0.02):
75
+ The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
76
+ rms_norm_eps (`float`, *optional*, defaults to 1e-06):
77
+ The epsilon used by the rms normalization layers.
78
+ use_cache (`bool`, *optional*, defaults to `True`):
79
+ Whether or not the model should return the last key/values attentions (not used by all models). Only
80
+ relevant if `config.is_decoder=True`.
81
+ pad_token_id (`int`, *optional*):
82
+ Padding token id.
83
+ bos_token_id (`int`, *optional*, defaults to 1):
84
+ Beginning of stream token id.
85
+ eos_token_id (`int`, *optional*, defaults to 2):
86
+ End of stream token id.
87
+ pretraining_tp (`int`, *optional*, defaults to 1):
88
+ Experimental feature. Tensor parallelism rank used during pretraining. Please refer to [this
89
+ document](https://huggingface.co/docs/transformers/parallelism) to understand more about it. This value is
90
+ necessary to ensure exact reproducibility of the pretraining results. Please refer to [this
91
+ issue](https://github.com/pytorch/pytorch/issues/76232).
92
+ tie_word_embeddings (`bool`, *optional*, defaults to `False`):
93
+ Whether to tie weight embeddings
94
+ rope_theta (`float`, *optional*, defaults to 10000.0):
95
+ The base period of the RoPE embeddings.
96
+ rope_scaling (`Dict`, *optional*):
97
+ Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports two scaling
98
+ strategies: linear and dynamic. Their scaling factor must be a float greater than 1. The expected format is
99
+ `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
100
+ `max_position_embeddings` to the expected new maximum.
101
+ attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`):
102
+ Whether to use a bias in the query, key, value and output projection layers during self-attention.
103
+ attention_dropout (`float`, *optional*, defaults to 0.0):
104
+ The dropout ratio for the attention probabilities.
105
+
106
+ ```python
107
+ >>> from transformers import DeepseekV3Model, DeepseekV3Config
108
+
109
+ >>> # Initializing a Deepseek-V3 style configuration
110
+ >>> configuration = DeepseekV3Config()
111
+
112
+ >>> # Accessing the model configuration
113
+ >>> configuration = model.config
114
+ ```"""
115
+
116
+ model_type = "deepseek_v3"
117
+ keys_to_ignore_at_inference = ["past_key_values"]
118
+
119
+ def __init__(
120
+ self,
121
+ vocab_size=129280,
122
+ hidden_size=7168,
123
+ intermediate_size=18432,
124
+ moe_intermediate_size = 2048,
125
+ num_hidden_layers=61,
126
+ num_nextn_predict_layers=1,
127
+ num_attention_heads=128,
128
+ num_key_value_heads=128,
129
+ n_shared_experts = 1,
130
+ n_routed_experts = 256,
131
+ ep_size = 1,
132
+ routed_scaling_factor = 2.5,
133
+ kv_lora_rank = 512,
134
+ q_lora_rank = 1536,
135
+ qk_rope_head_dim = 64,
136
+ v_head_dim = 128,
137
+ qk_nope_head_dim = 128,
138
+ topk_method = 'noaux_tc',
139
+ n_group = 8,
140
+ topk_group = 4,
141
+ num_experts_per_tok = 8,
142
+ moe_layer_freq = 1,
143
+ first_k_dense_replace = 3,
144
+ norm_topk_prob = True,
145
+ scoring_func = 'sigmoid',
146
+ aux_loss_alpha = 0.001,
147
+ seq_aux = True,
148
+ hidden_act="silu",
149
+ max_position_embeddings=4096,
150
+ initializer_range=0.02,
151
+ rms_norm_eps=1e-6,
152
+ use_cache=True,
153
+ pad_token_id=None,
154
+ bos_token_id=0,
155
+ eos_token_id=1,
156
+ pretraining_tp=1,
157
+ tie_word_embeddings=False,
158
+ rope_theta=10000.0,
159
+ rope_scaling=None,
160
+ attention_bias=False,
161
+ attention_dropout=0.0,
162
+ **kwargs,
163
+ ):
164
+ self.vocab_size = vocab_size
165
+ self.max_position_embeddings = max_position_embeddings
166
+ self.hidden_size = hidden_size
167
+ self.intermediate_size = intermediate_size
168
+ self.moe_intermediate_size = moe_intermediate_size
169
+ self.num_hidden_layers = num_hidden_layers
170
+ self.num_nextn_predict_layers = num_nextn_predict_layers
171
+ self.num_attention_heads = num_attention_heads
172
+ self.n_shared_experts = n_shared_experts
173
+ self.n_routed_experts = n_routed_experts
174
+ self.ep_size = ep_size
175
+ self.routed_scaling_factor = routed_scaling_factor
176
+ self.kv_lora_rank = kv_lora_rank
177
+ self.q_lora_rank = q_lora_rank
178
+ self.qk_rope_head_dim = qk_rope_head_dim
179
+ self.v_head_dim = v_head_dim
180
+ self.qk_nope_head_dim = qk_nope_head_dim
181
+ self.topk_method = topk_method
182
+ self.n_group = n_group
183
+ self.topk_group = topk_group
184
+ self.num_experts_per_tok = num_experts_per_tok
185
+ self.moe_layer_freq = moe_layer_freq
186
+ self.first_k_dense_replace = first_k_dense_replace
187
+ self.norm_topk_prob = norm_topk_prob
188
+ self.scoring_func = scoring_func
189
+ self.aux_loss_alpha = aux_loss_alpha
190
+ self.seq_aux = seq_aux
191
+ # for backward compatibility
192
+ if num_key_value_heads is None:
193
+ num_key_value_heads = num_attention_heads
194
+
195
+ self.num_key_value_heads = num_key_value_heads
196
+ self.hidden_act = hidden_act
197
+ self.initializer_range = initializer_range
198
+ self.rms_norm_eps = rms_norm_eps
199
+ self.pretraining_tp = pretraining_tp
200
+ self.use_cache = use_cache
201
+ self.rope_theta = rope_theta
202
+ self.rope_scaling = rope_scaling
203
+ self.attention_bias = attention_bias
204
+ self.attention_dropout = attention_dropout
205
+
206
+ super().__init__(
207
+ pad_token_id=pad_token_id,
208
+ bos_token_id=bos_token_id,
209
+ eos_token_id=eos_token_id,
210
+ tie_word_embeddings=tie_word_embeddings,
211
+ **kwargs,
212
+ )
generation_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_length": 131072,
3
+ "eos_token_id": 163585
4
+ }
kimi-logo.png ADDED
model-1-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba98e5298dedbc9ac9c4f9ad1d8074daef6cf8b89fc7b0f3a90deb5fea5f95c3
3
+ size 3343812312
model-10-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaabd42bdc33f62c30b28503dfbb3357a1c0fd83f3ec3d39c42ee9bd20fa7a4f
3
+ size 34118878784
model-11-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e75e5ba22fbe7432195d54a796222aa1a5973dcb5968dc01b4b782be2f8b567
3
+ size 34118879952
model-12-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1dbf9074a8a321542d67540d6b814618da25e15e8c7618ea198d866c59ae6259
3
+ size 34118879952
model-13-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21a02239a71f420796d9dccf37996d112a6ceca33dbbb374668807be9b73d673
3
+ size 34118879952
model-14-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1e9499c980a784fe5463e315525a61475f471eda18340e121efc3bf11f932e4
3
+ size 34118879952
model-15-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31af8b410886d0347931870866b06612ee82dc3341896cc625e55b7271a7b975
3
+ size 34118879952
model-16-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33f74b586c651251c2e0d78d095ce91b9f0a6b7d239d33566163224ecdd32a3c
3
+ size 34118879952
model-17-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cecd39c2ef157139d0a277a3a802da860c2f36904d3f6da80812c3e9c738b2e
3
+ size 34118879952
model-18-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f314b92cf222af98bd60c043c21429ed10db5366e58d38c702cc4ad86ecb0130
3
+ size 34118879952
model-19-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f73447db1348b72fce1b6427e31344f0e2ad16f8282988c77f0f3e998f280755
3
+ size 34118879952
model-2-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:450ca6404a46939e8131798846dcfacc41fd4550c063e7753652cb4c175867d1
3
+ size 34118878784
model-20-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1354554acdd1b910854c55b0f3a09cb468d6bd45780c02599771236d585db63
3
+ size 34118879952
model-21-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6a8a96c5f50df94eade87433adaa661a14ceb5b31b75d50719f17c2aed8dcc0
3
+ size 34118879952
model-22-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3dcf3c750c6e57471fef0e90a3f6196875651bf4c667276b9588782ba6fc98c
3
+ size 34118879952
model-23-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:512e072c34eeb6092ff248bd767971a2713edcc0cd82f0cef545e76c1261ca9b
3
+ size 34118879952
model-24-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38214a8aff2bf2927bb075dce46d4a013bba8702e18389b5e4c1ffd3eda95a68
3
+ size 34118879952
model-25-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdaca1b5caa29b3b9c8f7faf1835c0fb4a071ab8d33cb25e86bdf66aa34cf0d4
3
+ size 34118879952
model-26-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69d8efa04c7e93f98126678a8d53f6847457182ccd703fbb09832c90f79234ef
3
+ size 34118879952
model-27-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8997e9ecfc6a5fa18450ddb202953b93d03bcf4452d09e0609a97c89211199a8
3
+ size 34118879952
model-28-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:788cd02665538b24c84701f4bf82f3c38b9f1e8d326d17019ce4802b74edbeb1
3
+ size 34118879952
model-29-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4918004f7a337939271fcea91bcf9210bf795aef01057684170b510d0706ef6e
3
+ size 34118879952
model-3-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c44b0fc79854ce3567d942cec1e45d0e66a2118c1dafa091b9d0bf7bb5204df9
3
+ size 34118878784
model-30-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f372d714e9923bbe8aa889310d7a7982a7eb117e6bfc6ca334ea2b958d5e85f
3
+ size 34118879952
model-31-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b723978bb30d2ab3862a0149a74bb8d3c37e172773734c930b8a3b7d49576c8
3
+ size 34118879952
model-32-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:525565db30ad1c3ac5505befcf92a3bd18f25eab15fe9578e7bb0f33bcaa74ce
3
+ size 34118879952
model-33-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db3601256176633b87a2ff10b25a9a80f1d089a54399394ecd0bddacee01ad81
3
+ size 34118879952
model-34-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abde9fa95b6d85a5e949cbbca0de26f85fb04c33567f39e3477b1341b09c7037
3
+ size 34118879952
model-35-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcfa6e91e3b63f1c320bee4e5102f470e2316db10897273ddd5376bf2f24676d
3
+ size 34118879952
model-36-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fce47046a255d11b9c70caafefde7a9beeafa9b55fe40d86cfea2faa6c75c6d
3
+ size 34118879952
model-37-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f68f14d93eff9b3a723bae84b6c9d048bab9b5c3ef4f4b4697a4893fe4051d5
3
+ size 34118879952
model-38-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55a0a0c9c747f2588e4017e40f52d72241036e1b694e5a55e173a3879a2c185b
3
+ size 34118879952
model-39-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58e9568c2915f0efa32e9a74df6de1bd48ba4043fa71a35c0726697f50c77367
3
+ size 34118879952
model-4-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fd52afec92fc0140cb4826e671ca86548d66d97f5f461c896576b5997514014
3
+ size 34118878784
model-40-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82f900f1756bef273d35e640db1088936cba23a24dc41315172c79ba34b34461
3
+ size 34118879952
model-41-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aca4f38f08af51cc2ad6d9cd128e91e83ff491beeee1d869810fe93cc06ee501
3
+ size 34118879952
model-42-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43da2e62cc0953a3850f56e7bf94996461fb8416de93e950c1afbb502ca271fa
3
+ size 34118879952
model-43-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd015b2f74348ef895aa7dab8a15ab75bbab94ba77e092d31373341ca1b37050
3
+ size 34118879952
model-44-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:219b8f20837d2e6278d972235e7a70d0ab0ed036955b2ec39ca6711e9ce0e928
3
+ size 34118879952
model-45-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85e4c73774df0e6094117540ac7b31ce31926787cabc380ca42fd0ad8158e622
3
+ size 34118879952
model-46-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e145633b01dc41993e595f7cdbeb6a1f84ccd5bf4d8665750f4faadccc512eec
3
+ size 34118879952