Hugging Face's logo

yalhessi
/
lemexp-task1-template_small-Qwen2.5-1.5B-ddp-8lr

PEFT
Safetensors
Generated from Trainer
Model card Files Community
1

Improve language tag

#1
by lbourdois - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+135
-121
Files changed (1) hide show
  1. README.md +135 -121
README.md CHANGED
@@ -1,122 +1,136 @@
1
- ---
2
- library_name: peft
3
- license: apache-2.0
4
- base_model: Qwen/Qwen2.5-1.5B
5
- tags:
6
- - generated_from_trainer
7
- model-index:
8
- - name: lemexp-task1-template_small-Qwen2.5-1.5B-ddp-8lr
9
- results: []
10
- ---
11
-
12
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
- should probably proofread and complete it, then remove this comment. -->
14
-
15
- # lemexp-task1-template_small-Qwen2.5-1.5B-ddp-8lr
16
-
17
- This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on an unknown dataset.
18
- It achieves the following results on the evaluation set:
19
- - Loss: 0.3249
20
-
21
- ## Model description
22
-
23
- More information needed
24
-
25
- ## Intended uses & limitations
26
-
27
- More information needed
28
-
29
- ## Training and evaluation data
30
-
31
- More information needed
32
-
33
- ## Training procedure
34
-
35
- ### Training hyperparameters
36
-
37
- The following hyperparameters were used during training:
38
- - learning_rate: 0.0008
39
- - train_batch_size: 1
40
- - eval_batch_size: 2
41
- - seed: 42
42
- - distributed_type: multi-GPU
43
- - num_devices: 8
44
- - total_train_batch_size: 8
45
- - total_eval_batch_size: 16
46
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
47
- - lr_scheduler_type: linear
48
- - num_epochs: 12
49
- - mixed_precision_training: Native AMP
50
-
51
- ### Training results
52
-
53
- | Training Loss | Epoch | Step | Validation Loss |
54
- |:-------------:|:-------:|:-----:|:---------------:|
55
- | 0.5907 | 0.2001 | 1258 | 0.5642 |
56
- | 0.5363 | 0.4002 | 2516 | 0.5205 |
57
- | 0.521 | 0.6003 | 3774 | 0.4889 |
58
- | 0.5061 | 0.8004 | 5032 | 0.4806 |
59
- | 0.4959 | 1.0005 | 6290 | 0.4624 |
60
- | 0.4793 | 1.2006 | 7548 | 0.4559 |
61
- | 0.4771 | 1.4007 | 8806 | 0.4519 |
62
- | 0.4701 | 1.6008 | 10064 | 0.4431 |
63
- | 0.4734 | 1.8009 | 11322 | 0.4480 |
64
- | 0.4913 | 2.0010 | 12580 | 0.4428 |
65
- | 0.4523 | 2.2010 | 13838 | 0.4328 |
66
- | 0.4518 | 2.4011 | 15096 | 0.4327 |
67
- | 0.4503 | 2.6012 | 16354 | 0.4332 |
68
- | 0.4428 | 2.8013 | 17612 | 0.4187 |
69
- | 0.444 | 3.0014 | 18870 | 0.4209 |
70
- | 0.4286 | 3.2015 | 20128 | 0.4180 |
71
- | 0.4326 | 3.4016 | 21386 | 0.4132 |
72
- | 0.4311 | 3.6017 | 22644 | 0.4099 |
73
- | 0.4254 | 3.8018 | 23902 | 0.4069 |
74
- | 0.4231 | 4.0019 | 25160 | 0.3985 |
75
- | 0.4166 | 4.2020 | 26418 | 0.3994 |
76
- | 0.4127 | 4.4021 | 27676 | 0.3949 |
77
- | 0.4063 | 4.6022 | 28934 | 0.3907 |
78
- | 0.4089 | 4.8023 | 30192 | 0.3885 |
79
- | 0.4041 | 5.0024 | 31450 | 0.3907 |
80
- | 0.3926 | 5.2025 | 32708 | 0.3882 |
81
- | 0.39 | 5.4026 | 33966 | 0.3843 |
82
- | 0.3899 | 5.6027 | 35224 | 0.3794 |
83
- | 0.3902 | 5.8028 | 36482 | 0.3769 |
84
- | 0.3896 | 6.0029 | 37740 | 0.3720 |
85
- | 0.3781 | 6.2030 | 38998 | 0.3735 |
86
- | 0.3764 | 6.4031 | 40256 | 0.3683 |
87
- | 0.3719 | 6.6031 | 41514 | 0.3692 |
88
- | 0.3767 | 6.8032 | 42772 | 0.3648 |
89
- | 0.3745 | 7.0033 | 44030 | 0.3624 |
90
- | 0.3593 | 7.2034 | 45288 | 0.3636 |
91
- | 0.3603 | 7.4035 | 46546 | 0.3555 |
92
- | 0.3596 | 7.6036 | 47804 | 0.3522 |
93
- | 0.3567 | 7.8037 | 49062 | 0.3541 |
94
- | 0.3553 | 8.0038 | 50320 | 0.3514 |
95
- | 0.3427 | 8.2039 | 51578 | 0.3451 |
96
- | 0.3434 | 8.4040 | 52836 | 0.3480 |
97
- | 0.3465 | 8.6041 | 54094 | 0.3443 |
98
- | 0.3411 | 8.8042 | 55352 | 0.3435 |
99
- | 0.3402 | 9.0043 | 56610 | 0.3422 |
100
- | 0.3253 | 9.2044 | 57868 | 0.3404 |
101
- | 0.3251 | 9.4045 | 59126 | 0.3361 |
102
- | 0.3263 | 9.6046 | 60384 | 0.3355 |
103
- | 0.3258 | 9.8047 | 61642 | 0.3321 |
104
- | 0.3289 | 10.0048 | 62900 | 0.3315 |
105
- | 0.3093 | 10.2049 | 64158 | 0.3345 |
106
- | 0.3113 | 10.4050 | 65416 | 0.3326 |
107
- | 0.3084 | 10.6051 | 66674 | 0.3299 |
108
- | 0.3098 | 10.8052 | 67932 | 0.3277 |
109
- | 0.3064 | 11.0052 | 69190 | 0.3266 |
110
- | 0.2951 | 11.2053 | 70448 | 0.3289 |
111
- | 0.2951 | 11.4054 | 71706 | 0.3259 |
112
- | 0.2939 | 11.6055 | 72964 | 0.3255 |
113
- | 0.2923 | 11.8056 | 74222 | 0.3249 |
114
-
115
-
116
- ### Framework versions
117
-
118
- - PEFT 0.14.0
119
- - Transformers 4.47.0
120
- - Pytorch 2.5.1+cu124
121
- - Datasets 3.2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122
  - Tokenizers 0.21.0
 
1
+ ---
2
+ library_name: peft
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen2.5-1.5B
5
+ tags:
6
+ - generated_from_trainer
7
+ language:
8
+ - zho
9
+ - eng
10
+ - fra
11
+ - spa
12
+ - por
13
+ - deu
14
+ - ita
15
+ - rus
16
+ - jpn
17
+ - kor
18
+ - vie
19
+ - tha
20
+ - ara
21
+ model-index:
22
+ - name: lemexp-task1-template_small-Qwen2.5-1.5B-ddp-8lr
23
+ results: []
24
+ ---
25
+
26
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
27
+ should probably proofread and complete it, then remove this comment. -->
28
+
29
+ # lemexp-task1-template_small-Qwen2.5-1.5B-ddp-8lr
30
+
31
+ This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on an unknown dataset.
32
+ It achieves the following results on the evaluation set:
33
+ - Loss: 0.3249
34
+
35
+ ## Model description
36
+
37
+ More information needed
38
+
39
+ ## Intended uses & limitations
40
+
41
+ More information needed
42
+
43
+ ## Training and evaluation data
44
+
45
+ More information needed
46
+
47
+ ## Training procedure
48
+
49
+ ### Training hyperparameters
50
+
51
+ The following hyperparameters were used during training:
52
+ - learning_rate: 0.0008
53
+ - train_batch_size: 1
54
+ - eval_batch_size: 2
55
+ - seed: 42
56
+ - distributed_type: multi-GPU
57
+ - num_devices: 8
58
+ - total_train_batch_size: 8
59
+ - total_eval_batch_size: 16
60
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
61
+ - lr_scheduler_type: linear
62
+ - num_epochs: 12
63
+ - mixed_precision_training: Native AMP
64
+
65
+ ### Training results
66
+
67
+ | Training Loss | Epoch | Step | Validation Loss |
68
+ |:-------------:|:-------:|:-----:|:---------------:|
69
+ | 0.5907 | 0.2001 | 1258 | 0.5642 |
70
+ | 0.5363 | 0.4002 | 2516 | 0.5205 |
71
+ | 0.521 | 0.6003 | 3774 | 0.4889 |
72
+ | 0.5061 | 0.8004 | 5032 | 0.4806 |
73
+ | 0.4959 | 1.0005 | 6290 | 0.4624 |
74
+ | 0.4793 | 1.2006 | 7548 | 0.4559 |
75
+ | 0.4771 | 1.4007 | 8806 | 0.4519 |
76
+ | 0.4701 | 1.6008 | 10064 | 0.4431 |
77
+ | 0.4734 | 1.8009 | 11322 | 0.4480 |
78
+ | 0.4913 | 2.0010 | 12580 | 0.4428 |
79
+ | 0.4523 | 2.2010 | 13838 | 0.4328 |
80
+ | 0.4518 | 2.4011 | 15096 | 0.4327 |
81
+ | 0.4503 | 2.6012 | 16354 | 0.4332 |
82
+ | 0.4428 | 2.8013 | 17612 | 0.4187 |
83
+ | 0.444 | 3.0014 | 18870 | 0.4209 |
84
+ | 0.4286 | 3.2015 | 20128 | 0.4180 |
85
+ | 0.4326 | 3.4016 | 21386 | 0.4132 |
86
+ | 0.4311 | 3.6017 | 22644 | 0.4099 |
87
+ | 0.4254 | 3.8018 | 23902 | 0.4069 |
88
+ | 0.4231 | 4.0019 | 25160 | 0.3985 |
89
+ | 0.4166 | 4.2020 | 26418 | 0.3994 |
90
+ | 0.4127 | 4.4021 | 27676 | 0.3949 |
91
+ | 0.4063 | 4.6022 | 28934 | 0.3907 |
92
+ | 0.4089 | 4.8023 | 30192 | 0.3885 |
93
+ | 0.4041 | 5.0024 | 31450 | 0.3907 |
94
+ | 0.3926 | 5.2025 | 32708 | 0.3882 |
95
+ | 0.39 | 5.4026 | 33966 | 0.3843 |
96
+ | 0.3899 | 5.6027 | 35224 | 0.3794 |
97
+ | 0.3902 | 5.8028 | 36482 | 0.3769 |
98
+ | 0.3896 | 6.0029 | 37740 | 0.3720 |
99
+ | 0.3781 | 6.2030 | 38998 | 0.3735 |
100
+ | 0.3764 | 6.4031 | 40256 | 0.3683 |
101
+ | 0.3719 | 6.6031 | 41514 | 0.3692 |
102
+ | 0.3767 | 6.8032 | 42772 | 0.3648 |
103
+ | 0.3745 | 7.0033 | 44030 | 0.3624 |
104
+ | 0.3593 | 7.2034 | 45288 | 0.3636 |
105
+ | 0.3603 | 7.4035 | 46546 | 0.3555 |
106
+ | 0.3596 | 7.6036 | 47804 | 0.3522 |
107
+ | 0.3567 | 7.8037 | 49062 | 0.3541 |
108
+ | 0.3553 | 8.0038 | 50320 | 0.3514 |
109
+ | 0.3427 | 8.2039 | 51578 | 0.3451 |
110
+ | 0.3434 | 8.4040 | 52836 | 0.3480 |
111
+ | 0.3465 | 8.6041 | 54094 | 0.3443 |
112
+ | 0.3411 | 8.8042 | 55352 | 0.3435 |
113
+ | 0.3402 | 9.0043 | 56610 | 0.3422 |
114
+ | 0.3253 | 9.2044 | 57868 | 0.3404 |
115
+ | 0.3251 | 9.4045 | 59126 | 0.3361 |
116
+ | 0.3263 | 9.6046 | 60384 | 0.3355 |
117
+ | 0.3258 | 9.8047 | 61642 | 0.3321 |
118
+ | 0.3289 | 10.0048 | 62900 | 0.3315 |
119
+ | 0.3093 | 10.2049 | 64158 | 0.3345 |
120
+ | 0.3113 | 10.4050 | 65416 | 0.3326 |
121
+ | 0.3084 | 10.6051 | 66674 | 0.3299 |
122
+ | 0.3098 | 10.8052 | 67932 | 0.3277 |
123
+ | 0.3064 | 11.0052 | 69190 | 0.3266 |
124
+ | 0.2951 | 11.2053 | 70448 | 0.3289 |
125
+ | 0.2951 | 11.4054 | 71706 | 0.3259 |
126
+ | 0.2939 | 11.6055 | 72964 | 0.3255 |
127
+ | 0.2923 | 11.8056 | 74222 | 0.3249 |
128
+
129
+
130
+ ### Framework versions
131
+
132
+ - PEFT 0.14.0
133
+ - Transformers 4.47.0
134
+ - Pytorch 2.5.1+cu124
135
+ - Datasets 3.2.0
136
  - Tokenizers 0.21.0