Hugging Face's logo

zijianh
/
DeepSeek-R1-Distill-Qwen-7B-RL-length-penalty-low-new

Text Generation
Transformers
Safetensors
qwen2
Generated from Trainer
open-r1
trl
grpo
conversational
text-generation-inference
Model card Files Community
zijianh commited on
Commit
368616f
·
verified ·
1 Parent(s): 68a5fdc

Model save

Browse files
Files changed (5) hide show
  1. README.md +68 -0
  2. all_results.json +8 -0
  3. generation_config.json +9 -0
  4. train_results.json +8 -0
  5. trainer_state.json +418 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
3
+ library_name: transformers
4
+ model_name: DeepSeek-R1-Distill-Qwen-7B-RL-length-penalty-low-new
5
+ tags:
6
+ - generated_from_trainer
7
+ - trl
8
+ - grpo
9
+ licence: license
10
+ ---
11
+
12
+ # Model Card for DeepSeek-R1-Distill-Qwen-7B-RL-length-penalty-low-new
13
+
14
+ This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl).
16
+
17
+ ## Quick start
18
+
19
+ ```python
20
+ from transformers import pipeline
21
+
22
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="zijianh/DeepSeek-R1-Distill-Qwen-7B-RL-length-penalty-low-new", device="cuda")
24
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
+ print(output["generated_text"])
26
+ ```
27
+
28
+ ## Training procedure
29
+
30
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/sota_mavens-university-of-michigan/huggingface/runs/x6rhey2t)
31
+
32
+
33
+ This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
34
+
35
+ ### Framework versions
36
+
37
+ - TRL: 0.16.0.dev0
38
+ - Transformers: 4.49.0
39
+ - Pytorch: 2.5.1
40
+ - Datasets: 3.4.1
41
+ - Tokenizers: 0.21.1
42
+
43
+ ## Citations
44
+
45
+ Cite GRPO as:
46
+
47
+ ```bibtex
48
+ @article{zhihong2024deepseekmath,
49
+ title = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
50
+ author = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
51
+ year = 2024,
52
+ eprint = {arXiv:2402.03300},
53
+ }
54
+
55
+ ```
56
+
57
+ Cite TRL as:
58
+
59
+ ```bibtex
60
+ @misc{vonwerra2022trl,
61
+ title = {{TRL: Transformer Reinforcement Learning}},
62
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
63
+ year = 2020,
64
+ journal = {GitHub repository},
65
+ publisher = {GitHub},
66
+ howpublished = {\url{https://github.com/huggingface/trl}}
67
+ }
68
+ ```
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 0.0,
3
+ "train_loss": 0.019923711259817255,
4
+ "train_runtime": 38027.4532,
5
+ "train_samples": 7500,
6
+ "train_samples_per_second": 0.394,
7
+ "train_steps_per_second": 0.003
8
+ }
generation_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 151646,
4
+ "do_sample": true,
5
+ "eos_token_id": 151643,
6
+ "temperature": 0.6,
7
+ "top_p": 0.95,
8
+ "transformers_version": "4.49.0"
9
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 0.0,
3
+ "train_loss": 0.019923711259817255,
4
+ "train_runtime": 38027.4532,
5
+ "train_samples": 7500,
6
+ "train_samples_per_second": 0.394,
7
+ "train_steps_per_second": 0.003
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,418 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 1.9893390191897655,
5
+ "eval_steps": 100,
6
+ "global_step": 116,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "clip_ratio": 0.0,
13
+ "completion_length": 1228.861671447754,
14
+ "epoch": 0.017057569296375266,
15
+ "grad_norm": 0.06720272451639175,
16
+ "kl": 0.0,
17
+ "learning_rate": 2.5e-07,
18
+ "loss": 0.0434,
19
+ "reward": 0.6573660969734192,
20
+ "reward_std": 0.18419872969388962,
21
+ "rewards/accuracy_reward": 0.6573660969734192,
22
+ "rewards/format_reward": 0.0,
23
+ "rewards/len_penalty": 0.0,
24
+ "step": 1
25
+ },
26
+ {
27
+ "clip_ratio": 0.0,
28
+ "completion_length": 1225.2754497528076,
29
+ "epoch": 0.08528784648187633,
30
+ "grad_norm": 0.06428399682044983,
31
+ "kl": 0.00012925267219543457,
32
+ "learning_rate": 1.25e-06,
33
+ "loss": 0.0339,
34
+ "reward": 0.6021205559372902,
35
+ "reward_std": 0.16342801752034575,
36
+ "rewards/accuracy_reward": 0.6021205559372902,
37
+ "rewards/format_reward": 0.0,
38
+ "rewards/len_penalty": 0.0,
39
+ "step": 5
40
+ },
41
+ {
42
+ "clip_ratio": 0.0,
43
+ "completion_length": 1126.743356323242,
44
+ "epoch": 0.17057569296375266,
45
+ "grad_norm": 0.06763080507516861,
46
+ "kl": 0.00020864009857177735,
47
+ "learning_rate": 2.5e-06,
48
+ "loss": 0.0277,
49
+ "reward": 0.6676339641213417,
50
+ "reward_std": 0.1722445989958942,
51
+ "rewards/accuracy_reward": 0.6676339641213417,
52
+ "rewards/format_reward": 0.0,
53
+ "rewards/len_penalty": 0.0,
54
+ "step": 10
55
+ },
56
+ {
57
+ "clip_ratio": 0.0,
58
+ "completion_length": 1019.001612854004,
59
+ "epoch": 0.255863539445629,
60
+ "grad_norm": 0.06711237877607346,
61
+ "kl": 0.0011625289916992188,
62
+ "learning_rate": 2.9938448364256362e-06,
63
+ "loss": 0.0245,
64
+ "reward": 0.7089286029338837,
65
+ "reward_std": 0.16926544005982577,
66
+ "rewards/accuracy_reward": 0.7089286029338837,
67
+ "rewards/format_reward": 0.0,
68
+ "rewards/len_penalty": 0.0,
69
+ "step": 15
70
+ },
71
+ {
72
+ "clip_ratio": 0.0,
73
+ "completion_length": 911.8040542602539,
74
+ "epoch": 0.3411513859275053,
75
+ "grad_norm": 0.05857737362384796,
76
+ "kl": 0.0029880523681640623,
77
+ "learning_rate": 2.956412726139078e-06,
78
+ "loss": 0.0173,
79
+ "reward": 0.7337053924798965,
80
+ "reward_std": 0.1726918333210051,
81
+ "rewards/accuracy_reward": 0.7337053924798965,
82
+ "rewards/format_reward": 0.0,
83
+ "rewards/len_penalty": 0.0,
84
+ "step": 20
85
+ },
86
+ {
87
+ "clip_ratio": 0.0,
88
+ "completion_length": 943.9683486938477,
89
+ "epoch": 0.42643923240938164,
90
+ "grad_norm": 0.05815846845507622,
91
+ "kl": 0.003909683227539063,
92
+ "learning_rate": 2.88581929876693e-06,
93
+ "loss": 0.0094,
94
+ "reward": 0.7319196790456772,
95
+ "reward_std": 0.17025341242551803,
96
+ "rewards/accuracy_reward": 0.7319196790456772,
97
+ "rewards/format_reward": 0.0,
98
+ "rewards/len_penalty": 0.0,
99
+ "step": 25
100
+ },
101
+ {
102
+ "clip_ratio": 0.0,
103
+ "completion_length": 975.7393280029297,
104
+ "epoch": 0.511727078891258,
105
+ "grad_norm": 0.06264527887105942,
106
+ "kl": 0.006205368041992188,
107
+ "learning_rate": 2.7836719084521715e-06,
108
+ "loss": 0.0197,
109
+ "reward": 0.7424107506871224,
110
+ "reward_std": 0.17087422600015997,
111
+ "rewards/accuracy_reward": 0.7424107506871224,
112
+ "rewards/format_reward": 0.0,
113
+ "rewards/len_penalty": 0.0,
114
+ "step": 30
115
+ },
116
+ {
117
+ "clip_ratio": 0.0,
118
+ "completion_length": 959.3453567504882,
119
+ "epoch": 0.5970149253731343,
120
+ "grad_norm": 0.05786522477865219,
121
+ "kl": 0.006237030029296875,
122
+ "learning_rate": 2.652296367060421e-06,
123
+ "loss": 0.0126,
124
+ "reward": 0.7372768208384514,
125
+ "reward_std": 0.15754695991054177,
126
+ "rewards/accuracy_reward": 0.7372768208384514,
127
+ "rewards/format_reward": 0.0,
128
+ "rewards/len_penalty": 0.0,
129
+ "step": 35
130
+ },
131
+ {
132
+ "clip_ratio": 0.0,
133
+ "completion_length": 939.0549591064453,
134
+ "epoch": 0.6823027718550106,
135
+ "grad_norm": 0.0710252970457077,
136
+ "kl": 0.006754302978515625,
137
+ "learning_rate": 2.4946839873611927e-06,
138
+ "loss": 0.0152,
139
+ "reward": 0.7680803894996643,
140
+ "reward_std": 0.15489136478863658,
141
+ "rewards/accuracy_reward": 0.7680803894996643,
142
+ "rewards/format_reward": 0.0,
143
+ "rewards/len_penalty": 0.0,
144
+ "step": 40
145
+ },
146
+ {
147
+ "clip_ratio": 0.0,
148
+ "completion_length": 944.1554000854492,
149
+ "epoch": 0.767590618336887,
150
+ "grad_norm": 0.06381860375404358,
151
+ "kl": 0.007495880126953125,
152
+ "learning_rate": 2.314423473302218e-06,
153
+ "loss": 0.0181,
154
+ "reward": 0.7691964656114578,
155
+ "reward_std": 0.16262863762676716,
156
+ "rewards/accuracy_reward": 0.7691964656114578,
157
+ "rewards/format_reward": 0.0,
158
+ "rewards/len_penalty": 0.0,
159
+ "step": 45
160
+ },
161
+ {
162
+ "clip_ratio": 0.0,
163
+ "completion_length": 952.2839721679687,
164
+ "epoch": 0.8528784648187633,
165
+ "grad_norm": 0.0656137615442276,
166
+ "kl": 0.00856170654296875,
167
+ "learning_rate": 2.1156192081791355e-06,
168
+ "loss": 0.0141,
169
+ "reward": 0.7388393133878708,
170
+ "reward_std": 0.16094484115019442,
171
+ "rewards/accuracy_reward": 0.7388393133878708,
172
+ "rewards/format_reward": 0.0,
173
+ "rewards/len_penalty": 0.0,
174
+ "step": 50
175
+ },
176
+ {
177
+ "clip_ratio": 0.0,
178
+ "completion_length": 949.2908905029296,
179
+ "epoch": 0.9381663113006397,
180
+ "grad_norm": 0.061446413397789,
181
+ "kl": 0.00842742919921875,
182
+ "learning_rate": 1.9027978012115653e-06,
183
+ "loss": 0.017,
184
+ "reward": 0.768303607404232,
185
+ "reward_std": 0.15686179576441645,
186
+ "rewards/accuracy_reward": 0.768303607404232,
187
+ "rewards/format_reward": 0.0,
188
+ "rewards/len_penalty": 0.0,
189
+ "step": 55
190
+ },
191
+ {
192
+ "clip_ratio": 0.0,
193
+ "completion_length": 936.9299240112305,
194
+ "epoch": 1.0341151385927505,
195
+ "grad_norm": 0.06138957291841507,
196
+ "kl": 0.00886383056640625,
197
+ "learning_rate": 1.6808050203829845e-06,
198
+ "loss": 0.0141,
199
+ "reward": 0.7736607477068901,
200
+ "reward_std": 0.15758724519982933,
201
+ "rewards/accuracy_reward": 0.7736607477068901,
202
+ "rewards/format_reward": 0.0,
203
+ "rewards/len_penalty": 0.0,
204
+ "step": 60
205
+ },
206
+ {
207
+ "clip_ratio": 0.0,
208
+ "completion_length": 920.1882110595703,
209
+ "epoch": 1.1194029850746268,
210
+ "grad_norm": 0.063966765999794,
211
+ "kl": 0.009485626220703125,
212
+ "learning_rate": 1.454695458298667e-06,
213
+ "loss": 0.0157,
214
+ "reward": 0.7738839641213417,
215
+ "reward_std": 0.1468926408328116,
216
+ "rewards/accuracy_reward": 0.7738839641213417,
217
+ "rewards/format_reward": 0.0,
218
+ "rewards/len_penalty": 0.0,
219
+ "step": 65
220
+ },
221
+ {
222
+ "clip_ratio": 0.0,
223
+ "completion_length": 944.9875381469726,
224
+ "epoch": 1.2046908315565032,
225
+ "grad_norm": 0.08325506001710892,
226
+ "kl": 0.011992645263671876,
227
+ "learning_rate": 1.2296174432791415e-06,
228
+ "loss": 0.0216,
229
+ "reward": 0.7598214663565159,
230
+ "reward_std": 0.13879431886598467,
231
+ "rewards/accuracy_reward": 0.7598214663565159,
232
+ "rewards/format_reward": 0.0,
233
+ "rewards/len_penalty": 0.0,
234
+ "step": 70
235
+ },
236
+ {
237
+ "clip_ratio": 0.0,
238
+ "completion_length": 947.0911117553711,
239
+ "epoch": 1.2899786780383795,
240
+ "grad_norm": 0.0815800353884697,
241
+ "kl": 0.01367034912109375,
242
+ "learning_rate": 1.0106958161686963e-06,
243
+ "loss": 0.0215,
244
+ "reward": 0.759375037252903,
245
+ "reward_std": 0.15064039630815387,
246
+ "rewards/accuracy_reward": 0.759375037252903,
247
+ "rewards/format_reward": 0.0,
248
+ "rewards/len_penalty": 0.0,
249
+ "step": 75
250
+ },
251
+ {
252
+ "clip_ratio": 0.0,
253
+ "completion_length": 918.9600845336914,
254
+ "epoch": 1.375266524520256,
255
+ "grad_norm": 0.08615086227655411,
256
+ "kl": 0.019728851318359376,
257
+ "learning_rate": 8.029152419343472e-07,
258
+ "loss": 0.0195,
259
+ "reward": 0.7761161088943481,
260
+ "reward_std": 0.1427250973880291,
261
+ "rewards/accuracy_reward": 0.7761161088943481,
262
+ "rewards/format_reward": 0.0,
263
+ "rewards/len_penalty": 0.0,
264
+ "step": 80
265
+ },
266
+ {
267
+ "clip_ratio": 0.0,
268
+ "completion_length": 926.689549255371,
269
+ "epoch": 1.4605543710021323,
270
+ "grad_norm": 0.08062485605478287,
271
+ "kl": 0.01549224853515625,
272
+ "learning_rate": 6.11006712953975e-07,
273
+ "loss": 0.0165,
274
+ "reward": 0.755580386519432,
275
+ "reward_std": 0.1542196120135486,
276
+ "rewards/accuracy_reward": 0.755580386519432,
277
+ "rewards/format_reward": 0.0,
278
+ "rewards/len_penalty": 0.0,
279
+ "step": 85
280
+ },
281
+ {
282
+ "clip_ratio": 0.0,
283
+ "completion_length": 951.833529663086,
284
+ "epoch": 1.5458422174840085,
285
+ "grad_norm": 0.08391877263784409,
286
+ "kl": 0.01885833740234375,
287
+ "learning_rate": 4.3933982822017883e-07,
288
+ "loss": 0.0217,
289
+ "reward": 0.745089316368103,
290
+ "reward_std": 0.15332454843446613,
291
+ "rewards/accuracy_reward": 0.745089316368103,
292
+ "rewards/format_reward": 0.0,
293
+ "rewards/len_penalty": 0.0,
294
+ "step": 90
295
+ },
296
+ {
297
+ "clip_ratio": 0.0,
298
+ "completion_length": 961.7170059204102,
299
+ "epoch": 1.6311300639658848,
300
+ "grad_norm": 0.10089623183012009,
301
+ "kl": 0.02075958251953125,
302
+ "learning_rate": 2.9182330117358096e-07,
303
+ "loss": 0.026,
304
+ "reward": 0.7386161059141159,
305
+ "reward_std": 0.17298963768407702,
306
+ "rewards/accuracy_reward": 0.7386161059141159,
307
+ "rewards/format_reward": 0.0,
308
+ "rewards/len_penalty": 0.0,
309
+ "step": 95
310
+ },
311
+ {
312
+ "epoch": 1.716417910447761,
313
+ "grad_norm": 0.08750782907009125,
314
+ "learning_rate": 1.718159615201853e-07,
315
+ "loss": 0.0169,
316
+ "step": 100
317
+ },
318
+ {
319
+ "epoch": 1.716417910447761,
320
+ "eval_clip_ratio": 0.0,
321
+ "eval_completion_length": 908.4477716512954,
322
+ "eval_kl": 0.01996454110922524,
323
+ "eval_loss": 0.018831240013241768,
324
+ "eval_reward": 0.7642058762498557,
325
+ "eval_reward_std": 0.16209243547421293,
326
+ "eval_rewards/accuracy_reward": 0.7642058762498557,
327
+ "eval_rewards/format_reward": 0.0,
328
+ "eval_rewards/len_penalty": 0.0,
329
+ "eval_runtime": 7276.3561,
330
+ "eval_samples_per_second": 0.687,
331
+ "eval_steps_per_second": 0.006,
332
+ "step": 100
333
+ },
334
+ {
335
+ "clip_ratio": 0.0,
336
+ "completion_length": 948.437540435791,
337
+ "epoch": 1.8017057569296375,
338
+ "grad_norm": 0.07978012412786484,
339
+ "kl": 0.0216644287109375,
340
+ "learning_rate": 8.20502774480395e-08,
341
+ "loss": 0.0227,
342
+ "reward": 0.7496652126312255,
343
+ "reward_std": 0.1498533170670271,
344
+ "rewards/accuracy_reward": 0.7496652126312255,
345
+ "rewards/format_reward": 0.0,
346
+ "rewards/len_penalty": 0.0,
347
+ "step": 105
348
+ },
349
+ {
350
+ "clip_ratio": 0.0,
351
+ "completion_length": 941.6007141113281,
352
+ "epoch": 1.886993603411514,
353
+ "grad_norm": 0.08210857957601547,
354
+ "kl": 0.02252197265625,
355
+ "learning_rate": 2.4570139579284723e-08,
356
+ "loss": 0.0152,
357
+ "reward": 0.7466518208384514,
358
+ "reward_std": 0.1587499282322824,
359
+ "rewards/accuracy_reward": 0.7466518208384514,
360
+ "rewards/format_reward": 0.0,
361
+ "rewards/len_penalty": 0.0,
362
+ "step": 110
363
+ },
364
+ {
365
+ "clip_ratio": 0.0,
366
+ "completion_length": 948.6848648071289,
367
+ "epoch": 1.9722814498933903,
368
+ "grad_norm": 0.10769649595022202,
369
+ "kl": 0.02285919189453125,
370
+ "learning_rate": 6.843232656998933e-10,
371
+ "loss": 0.0252,
372
+ "reward": 0.750669677555561,
373
+ "reward_std": 0.16108589163050055,
374
+ "rewards/accuracy_reward": 0.750669677555561,
375
+ "rewards/format_reward": 0.0,
376
+ "rewards/len_penalty": 0.0,
377
+ "step": 115
378
+ },
379
+ {
380
+ "clip_ratio": 0.0,
381
+ "completion_length": 983.2754592895508,
382
+ "epoch": 1.9893390191897655,
383
+ "kl": 0.02458953857421875,
384
+ "reward": 0.7120535969734192,
385
+ "reward_std": 0.1434549919795245,
386
+ "rewards/accuracy_reward": 0.7120535969734192,
387
+ "rewards/format_reward": 0.0,
388
+ "rewards/len_penalty": 0.0,
389
+ "step": 116,
390
+ "total_flos": 0.0,
391
+ "train_loss": 0.019923711259817255,
392
+ "train_runtime": 38027.4532,
393
+ "train_samples_per_second": 0.394,
394
+ "train_steps_per_second": 0.003
395
+ }
396
+ ],
397
+ "logging_steps": 5,
398
+ "max_steps": 116,
399
+ "num_input_tokens_seen": 0,
400
+ "num_train_epochs": 2,
401
+ "save_steps": 10,
402
+ "stateful_callbacks": {
403
+ "TrainerControl": {
404
+ "args": {
405
+ "should_epoch_stop": false,
406
+ "should_evaluate": false,
407
+ "should_log": false,
408
+ "should_save": true,
409
+ "should_training_stop": true
410
+ },
411
+ "attributes": {}
412
+ }
413
+ },
414
+ "total_flos": 0.0,
415
+ "train_batch_size": 16,
416
+ "trial_name": null,
417
+ "trial_params": null
418
+ }