Hugging Face's logo

CleitonOERocha
/
vision-transformer-glomerulus-classifier

Image Classification
Transformers
TensorBoard
Safetensors
PyTorch
vit
vision
medical
glomerulus
pathology
vision-transformer
Eval Results
Model card Files Metrics Community
CleitonOERocha commited on
Commit
80c1f4a
·
verified ·
1 Parent(s): d6e5756

Upload do Vision Transformer para classificação de glomérulos

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +140 -0
  2. checkpoint-1038/config.json +49 -0
  3. checkpoint-1038/model.safetensors +3 -0
  4. checkpoint-1038/optimizer.pt +3 -0
  5. checkpoint-1038/preprocessor_config.json +23 -0
  6. checkpoint-1038/rng_state.pth +3 -0
  7. checkpoint-1038/scaler.pt +3 -0
  8. checkpoint-1038/scheduler.pt +3 -0
  9. checkpoint-1038/trainer_state.json +246 -0
  10. checkpoint-1038/training_args.bin +3 -0
  11. checkpoint-173/config.json +49 -0
  12. checkpoint-173/model.safetensors +3 -0
  13. checkpoint-173/optimizer.pt +3 -0
  14. checkpoint-173/preprocessor_config.json +23 -0
  15. checkpoint-173/rng_state.pth +3 -0
  16. checkpoint-173/scaler.pt +3 -0
  17. checkpoint-173/scheduler.pt +3 -0
  18. checkpoint-173/trainer_state.json +67 -0
  19. checkpoint-173/training_args.bin +3 -0
  20. checkpoint-346/config.json +49 -0
  21. checkpoint-346/model.safetensors +3 -0
  22. checkpoint-346/optimizer.pt +3 -0
  23. checkpoint-346/preprocessor_config.json +23 -0
  24. checkpoint-346/rng_state.pth +3 -0
  25. checkpoint-346/scaler.pt +3 -0
  26. checkpoint-346/scheduler.pt +3 -0
  27. checkpoint-346/trainer_state.json +100 -0
  28. checkpoint-346/training_args.bin +3 -0
  29. checkpoint-519/config.json +49 -0
  30. checkpoint-519/model.safetensors +3 -0
  31. checkpoint-519/optimizer.pt +3 -0
  32. checkpoint-519/preprocessor_config.json +23 -0
  33. checkpoint-519/rng_state.pth +3 -0
  34. checkpoint-519/scaler.pt +3 -0
  35. checkpoint-519/scheduler.pt +3 -0
  36. checkpoint-519/trainer_state.json +140 -0
  37. checkpoint-519/training_args.bin +3 -0
  38. checkpoint-692/config.json +49 -0
  39. checkpoint-692/model.safetensors +3 -0
  40. checkpoint-692/optimizer.pt +3 -0
  41. checkpoint-692/preprocessor_config.json +23 -0
  42. checkpoint-692/rng_state.pth +3 -0
  43. checkpoint-692/scaler.pt +3 -0
  44. checkpoint-692/scheduler.pt +3 -0
  45. checkpoint-692/trainer_state.json +173 -0
  46. checkpoint-692/training_args.bin +3 -0
  47. checkpoint-865/config.json +49 -0
  48. checkpoint-865/model.safetensors +3 -0
  49. checkpoint-865/optimizer.pt +3 -0
  50. checkpoint-865/preprocessor_config.json +23 -0
README.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - vision
5
+ - image-classification
6
+ - medical
7
+ - glomerulus
8
+ - pathology
9
+ - pytorch
10
+ - transformers
11
+ - vision-transformer
12
+ datasets:
13
+ - custom
14
+ metrics:
15
+ - accuracy
16
+ - f1
17
+ model-index:
18
+ - name: vision-transformer-glomerulus-classifier
19
+ results:
20
+ - task:
21
+ type: image-classification
22
+ name: Image Classification
23
+ dataset:
24
+ name: Glomerulus Classification Dataset
25
+ type: custom
26
+ metrics:
27
+ - type: accuracy
28
+ value: 0.9491
29
+ - type: f1
30
+ value: 0.9493
31
+ ---
32
+
33
+ # Vision Transformer for Glomerulus Classification
34
+
35
+ This model is a **Vision Transformer Large** fine-tuned for classification of renal glomerulus images, distinguishing between normal and crescentic conditions across different histological stains.
36
+
37
+ ## Model Performance
38
+
39
+ - **Accuracy**: 0.9491 (94.91%)
40
+ - **F1-Score**: 0.9493 (94.93%)
41
+
42
+
43
+ ## Supported Classes
44
+
45
+ The model classifies glomeruli into **10 classes**:
46
+
47
+ ### Normal:
48
+ - `Normal_AZAN` - Normal glomerulus with Azan staining
49
+ - `Normal_HE` - Normal glomerulus with H&E staining
50
+ - `Normal_PAMS` - Normal glomerulus with PAMS staining
51
+ - `Normal_PAS` - Normal glomerulus with PAS staining
52
+ - `Normal_PICRO` - Normal glomerulus with Picro staining
53
+
54
+ ### Crescentic:
55
+ - `Crescente_AZAN` - Crescentic glomerulus with Azan staining
56
+ - `Crescente_HE` - Crescentic glomerulus with H&E staining
57
+ - `Crescente_PAMS` - Crescentic glomerulus with PAMS staining
58
+ - `Crescente_PAS` - Crescentic glomerulus with PAS staining
59
+ - `Crescente_PICRO` - Crescentic glomerulus with Picro staining
60
+
61
+ ## How to Use
62
+
63
+ ```python
64
+ from transformers import AutoImageProcessor, AutoModelForImageClassification
65
+ from PIL import Image
66
+ import torch
67
+
68
+ # Load model and processor
69
+ model_name = "CleitonOERocha/vision-transformer-glomerulus-classifier"
70
+ processor = AutoImageProcessor.from_pretrained(model_name)
71
+ model = AutoModelForImageClassification.from_pretrained(model_name)
72
+
73
+ # Load and process image
74
+ image = Image.open("your_image.jpg")
75
+ inputs = processor(images=image, return_tensors="pt")
76
+
77
+ # Make prediction
78
+ with torch.no_grad():
79
+ outputs = model(**inputs)
80
+ predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
81
+ predicted_class_id = predictions.argmax().item()
82
+ predicted_class = model.config.id2label[predicted_class_id]
83
+ confidence = predictions.max().item()
84
+
85
+ print(f"Predicted class: {predicted_class}")
86
+ print(f"Confidence: {confidence:.4f}")
87
+ ```
88
+
89
+ ## Technical Details
90
+
91
+ - **Architecture**: Vision Transformer Large (ViT-L/16)
92
+ - **Base Model**: google/vit-large-patch16-224-in21k
93
+ - **Input Size**: 224x224 pixels
94
+ - **Patch Size**: 16x16
95
+ - **Parameters**: ~300M
96
+ - **Framework**: PyTorch + Transformers
97
+
98
+ ## Model Comparison
99
+
100
+ In our benchmark:
101
+ 1. **Vision Transformer Large**: 94.91% (this model)
102
+ 2. **Vision Transformer Base**: 93.53%
103
+ 3. **ResNet-50**: 50.58%
104
+
105
+ ## Performance by Class
106
+
107
+ | Class | Precision | Recall | F1-Score | Support |
108
+ |-------|-----------|--------|----------|---------|
109
+ | Crescente_AZAN | 0.9500 | 0.9344 | 0.9421 | 61 |
110
+ | Crescente_HE | 0.9665 | 0.9914 | 0.9788 | 233 |
111
+ | Crescente_PAMS | 0.9737 | 0.9487 | 0.9610 | 78 |
112
+ | Crescente_PAS | 0.9607 | 0.9661 | 0.9634 | 177 |
113
+ | Crescente_PICRO | 0.7179 | 0.6364 | 0.6747 | 44 |
114
+ | Normal_AZAN | 0.8714 | 0.9683 | 0.9173 | 63 |
115
+ | Normal_HE | 0.9908 | 0.9908 | 0.9908 | 434 |
116
+ | Normal_PAMS | 0.9785 | 0.9681 | 0.9733 | 94 |
117
+ | Normal_PAS | 0.9835 | 0.8151 | 0.8914 | 146 |
118
+ | Normal_PICRO | 0.6667 | 0.9565 | 0.7857 | 46 |
119
+
120
+ ## Dataset
121
+
122
+ The model was trained on a custom dataset of renal glomerulus images with:
123
+ - **Total images**: 2,759 images
124
+ - **Training**: 1,383 images
125
+ - **Test**: 1,376 images
126
+ - **Stains**: 5 types (AZAN, H&E, PAMS, PAS, PICRO)
127
+ - **Conditions**: Normal vs Crescentic
128
+
129
+
130
+ ## Limitations
131
+
132
+ - Trained specifically for renal glomeruli
133
+ - Requires histological quality images
134
+ - Does not replace professional medical diagnosis
135
+ - Performance may vary with different equipment/protocols
136
+
137
+
138
+ ---
139
+
140
+ **Note**: This model is for research and educational purposes. It should not be used as the sole source for medical diagnosis.
checkpoint-1038/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-1038/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5c7af14bfd69289e47fb95ed25213a42a35b9390c5ab388bd3ff39b13208ac8
3
+ size 1213294072
checkpoint-1038/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f18666a29d1f264a4d9e95ff8042089b3192d69d742c76432b5310ca95acb76
3
+ size 2426823082
checkpoint-1038/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }
checkpoint-1038/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2b5691046896d865f067a1958689168fc2411c74d2f82d596bd6a636b2b141b
3
+ size 14244
checkpoint-1038/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:905cade0296cc9a60e8224f0659abd3dffd5117fba6947f896f66b7af1f0fe94
3
+ size 988
checkpoint-1038/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:861158475a467c51c6d851a50f7818583cfa5a3dd0c7232c7434c41167ef1d18
3
+ size 1064
checkpoint-1038/trainer_state.json ADDED
@@ -0,0 +1,246 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 519,
3
+ "best_metric": 0.9491279069767442,
4
+ "best_model_checkpoint": "./results_google_vit-large-patch16-224-in21k/checkpoint-519",
5
+ "epoch": 6.0,
6
+ "eval_steps": 500,
7
+ "global_step": 1038,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.28901734104046245,
14
+ "grad_norm": 5.746753215789795,
15
+ "learning_rate": 2.45e-05,
16
+ "loss": 1.5215,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.5780346820809249,
21
+ "grad_norm": 2.093648672103882,
22
+ "learning_rate": 4.9500000000000004e-05,
23
+ "loss": 0.4825,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.8670520231213873,
28
+ "grad_norm": 3.6535494327545166,
29
+ "learning_rate": 4.738805970149254e-05,
30
+ "loss": 0.3352,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 1.0,
35
+ "eval_accuracy": 0.8997093023255814,
36
+ "eval_f1": 0.9002075736375416,
37
+ "eval_loss": 0.3576313555240631,
38
+ "eval_precision": 0.905387724540619,
39
+ "eval_recall": 0.8997093023255814,
40
+ "eval_runtime": 36.7463,
41
+ "eval_samples_per_second": 37.446,
42
+ "eval_steps_per_second": 4.681,
43
+ "step": 173
44
+ },
45
+ {
46
+ "epoch": 1.1560693641618498,
47
+ "grad_norm": 0.14127115905284882,
48
+ "learning_rate": 4.47228144989339e-05,
49
+ "loss": 0.1697,
50
+ "step": 200
51
+ },
52
+ {
53
+ "epoch": 1.4450867052023122,
54
+ "grad_norm": 0.16555330157279968,
55
+ "learning_rate": 4.2057569296375274e-05,
56
+ "loss": 0.2157,
57
+ "step": 250
58
+ },
59
+ {
60
+ "epoch": 1.7341040462427746,
61
+ "grad_norm": 8.537938117980957,
62
+ "learning_rate": 3.9392324093816635e-05,
63
+ "loss": 0.1304,
64
+ "step": 300
65
+ },
66
+ {
67
+ "epoch": 2.0,
68
+ "eval_accuracy": 0.9295058139534884,
69
+ "eval_f1": 0.9299410884859952,
70
+ "eval_loss": 0.2657730281352997,
71
+ "eval_precision": 0.933292642170999,
72
+ "eval_recall": 0.9295058139534884,
73
+ "eval_runtime": 36.4406,
74
+ "eval_samples_per_second": 37.76,
75
+ "eval_steps_per_second": 4.72,
76
+ "step": 346
77
+ },
78
+ {
79
+ "epoch": 2.023121387283237,
80
+ "grad_norm": 0.20084255933761597,
81
+ "learning_rate": 3.6727078891258e-05,
82
+ "loss": 0.1167,
83
+ "step": 350
84
+ },
85
+ {
86
+ "epoch": 2.3121387283236996,
87
+ "grad_norm": 0.020546281710267067,
88
+ "learning_rate": 3.4061833688699365e-05,
89
+ "loss": 0.0301,
90
+ "step": 400
91
+ },
92
+ {
93
+ "epoch": 2.601156069364162,
94
+ "grad_norm": 0.38922008872032166,
95
+ "learning_rate": 3.139658848614073e-05,
96
+ "loss": 0.0882,
97
+ "step": 450
98
+ },
99
+ {
100
+ "epoch": 2.8901734104046244,
101
+ "grad_norm": 0.017280207946896553,
102
+ "learning_rate": 2.8731343283582092e-05,
103
+ "loss": 0.075,
104
+ "step": 500
105
+ },
106
+ {
107
+ "epoch": 3.0,
108
+ "eval_accuracy": 0.9491279069767442,
109
+ "eval_f1": 0.9493183237288035,
110
+ "eval_loss": 0.23121964931488037,
111
+ "eval_precision": 0.9533860558309619,
112
+ "eval_recall": 0.9491279069767442,
113
+ "eval_runtime": 36.6312,
114
+ "eval_samples_per_second": 37.564,
115
+ "eval_steps_per_second": 4.695,
116
+ "step": 519
117
+ },
118
+ {
119
+ "epoch": 3.179190751445087,
120
+ "grad_norm": 0.01825469546020031,
121
+ "learning_rate": 2.6066098081023454e-05,
122
+ "loss": 0.0221,
123
+ "step": 550
124
+ },
125
+ {
126
+ "epoch": 3.468208092485549,
127
+ "grad_norm": 0.3856063485145569,
128
+ "learning_rate": 2.345415778251599e-05,
129
+ "loss": 0.0156,
130
+ "step": 600
131
+ },
132
+ {
133
+ "epoch": 3.7572254335260116,
134
+ "grad_norm": 0.024169214069843292,
135
+ "learning_rate": 2.0788912579957357e-05,
136
+ "loss": 0.0384,
137
+ "step": 650
138
+ },
139
+ {
140
+ "epoch": 4.0,
141
+ "eval_accuracy": 0.940406976744186,
142
+ "eval_f1": 0.9397796560818565,
143
+ "eval_loss": 0.25720012187957764,
144
+ "eval_precision": 0.9423321815760374,
145
+ "eval_recall": 0.940406976744186,
146
+ "eval_runtime": 36.5035,
147
+ "eval_samples_per_second": 37.695,
148
+ "eval_steps_per_second": 4.712,
149
+ "step": 692
150
+ },
151
+ {
152
+ "epoch": 4.046242774566474,
153
+ "grad_norm": 0.011584389954805374,
154
+ "learning_rate": 1.812366737739872e-05,
155
+ "loss": 0.004,
156
+ "step": 700
157
+ },
158
+ {
159
+ "epoch": 4.335260115606936,
160
+ "grad_norm": 0.02991395629942417,
161
+ "learning_rate": 1.5458422174840087e-05,
162
+ "loss": 0.0027,
163
+ "step": 750
164
+ },
165
+ {
166
+ "epoch": 4.624277456647399,
167
+ "grad_norm": 0.014115595258772373,
168
+ "learning_rate": 1.2793176972281452e-05,
169
+ "loss": 0.0047,
170
+ "step": 800
171
+ },
172
+ {
173
+ "epoch": 4.913294797687861,
174
+ "grad_norm": 0.04197293147444725,
175
+ "learning_rate": 1.0127931769722815e-05,
176
+ "loss": 0.0017,
177
+ "step": 850
178
+ },
179
+ {
180
+ "epoch": 5.0,
181
+ "eval_accuracy": 0.9433139534883721,
182
+ "eval_f1": 0.9428572613065572,
183
+ "eval_loss": 0.23874278366565704,
184
+ "eval_precision": 0.9446283352762681,
185
+ "eval_recall": 0.9433139534883721,
186
+ "eval_runtime": 36.7069,
187
+ "eval_samples_per_second": 37.486,
188
+ "eval_steps_per_second": 4.686,
189
+ "step": 865
190
+ },
191
+ {
192
+ "epoch": 5.202312138728324,
193
+ "grad_norm": 0.01218091044574976,
194
+ "learning_rate": 7.4626865671641785e-06,
195
+ "loss": 0.0015,
196
+ "step": 900
197
+ },
198
+ {
199
+ "epoch": 5.491329479768786,
200
+ "grad_norm": 0.016713842749595642,
201
+ "learning_rate": 4.797441364605544e-06,
202
+ "loss": 0.0014,
203
+ "step": 950
204
+ },
205
+ {
206
+ "epoch": 5.780346820809249,
207
+ "grad_norm": 0.020327085629105568,
208
+ "learning_rate": 2.132196162046908e-06,
209
+ "loss": 0.0013,
210
+ "step": 1000
211
+ },
212
+ {
213
+ "epoch": 6.0,
214
+ "eval_accuracy": 0.9440406976744186,
215
+ "eval_f1": 0.9436455460249632,
216
+ "eval_loss": 0.23752979934215546,
217
+ "eval_precision": 0.9451293374988102,
218
+ "eval_recall": 0.9440406976744186,
219
+ "eval_runtime": 36.5702,
220
+ "eval_samples_per_second": 37.626,
221
+ "eval_steps_per_second": 4.703,
222
+ "step": 1038
223
+ }
224
+ ],
225
+ "logging_steps": 50,
226
+ "max_steps": 1038,
227
+ "num_input_tokens_seen": 0,
228
+ "num_train_epochs": 6,
229
+ "save_steps": 500,
230
+ "stateful_callbacks": {
231
+ "TrainerControl": {
232
+ "args": {
233
+ "should_epoch_stop": false,
234
+ "should_evaluate": false,
235
+ "should_log": false,
236
+ "should_save": true,
237
+ "should_training_stop": true
238
+ },
239
+ "attributes": {}
240
+ }
241
+ },
242
+ "total_flos": 2.2731672793183764e+18,
243
+ "train_batch_size": 8,
244
+ "trial_name": null,
245
+ "trial_params": null
246
+ }
checkpoint-1038/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6c9c3f17910ac792342cf93f2277214bda780f04d3229697f0bcc6d445c55e
3
+ size 5432
checkpoint-173/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-173/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44a09316448c9265a8d4ddd4c6094a6e5eff6df8419c0017547096bf758fce30
3
+ size 1213294072
checkpoint-173/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3f34d86134f92c015ea3f80cbac02f920d6cc52cdee59567ef1e2c4f98ed6e2
3
+ size 2426823082
checkpoint-173/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }
checkpoint-173/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b66e3cc7c452b707ddac5caf0aa17618afb9bc1a0333600a22c4afb353f3165
3
+ size 14244
checkpoint-173/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbb3f2c7a115a02124fc26d3f3e47bfa94d4385be966a8ed53f1cc8372900c6f
3
+ size 988
checkpoint-173/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bdb9d2ab2c7d182d8d6cf51320dacc2dcfc9cf824d4ad4dce15b3dab81e4ea6d
3
+ size 1064
checkpoint-173/trainer_state.json ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 173,
3
+ "best_metric": 0.8997093023255814,
4
+ "best_model_checkpoint": "./results_google_vit-large-patch16-224-in21k/checkpoint-173",
5
+ "epoch": 1.0,
6
+ "eval_steps": 500,
7
+ "global_step": 173,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.28901734104046245,
14
+ "grad_norm": 5.746753215789795,
15
+ "learning_rate": 2.45e-05,
16
+ "loss": 1.5215,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.5780346820809249,
21
+ "grad_norm": 2.093648672103882,
22
+ "learning_rate": 4.9500000000000004e-05,
23
+ "loss": 0.4825,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.8670520231213873,
28
+ "grad_norm": 3.6535494327545166,
29
+ "learning_rate": 4.738805970149254e-05,
30
+ "loss": 0.3352,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 1.0,
35
+ "eval_accuracy": 0.8997093023255814,
36
+ "eval_f1": 0.9002075736375416,
37
+ "eval_loss": 0.3576313555240631,
38
+ "eval_precision": 0.905387724540619,
39
+ "eval_recall": 0.8997093023255814,
40
+ "eval_runtime": 36.7463,
41
+ "eval_samples_per_second": 37.446,
42
+ "eval_steps_per_second": 4.681,
43
+ "step": 173
44
+ }
45
+ ],
46
+ "logging_steps": 50,
47
+ "max_steps": 1038,
48
+ "num_input_tokens_seen": 0,
49
+ "num_train_epochs": 6,
50
+ "save_steps": 500,
51
+ "stateful_callbacks": {
52
+ "TrainerControl": {
53
+ "args": {
54
+ "should_epoch_stop": false,
55
+ "should_evaluate": false,
56
+ "should_log": false,
57
+ "should_save": true,
58
+ "should_training_stop": false
59
+ },
60
+ "attributes": {}
61
+ }
62
+ },
63
+ "total_flos": 3.788612132197294e+17,
64
+ "train_batch_size": 8,
65
+ "trial_name": null,
66
+ "trial_params": null
67
+ }
checkpoint-173/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6c9c3f17910ac792342cf93f2277214bda780f04d3229697f0bcc6d445c55e
3
+ size 5432
checkpoint-346/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-346/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16baea7054116b50589edf08238b67c09bcdb44102632134ed8fa700bb57e684
3
+ size 1213294072
checkpoint-346/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49c7644f301a23d5ea9adf2429cd815d7c1094dc513fab3b98b664537e25e167
3
+ size 2426823082
checkpoint-346/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }
checkpoint-346/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f37c40ce327861a7ca13b719d3aa37510a143368b6e74358bdb14becb3899e1e
3
+ size 14244
checkpoint-346/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64fde29b279e6b3aa49d19d3ea139f3ca54384174f4fc03279fae226afc19459
3
+ size 988
checkpoint-346/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de6bdb1c04016adfccb0cfe9cc0657915cf966ddcd96e5bda3c28b053b71d3ad
3
+ size 1064
checkpoint-346/trainer_state.json ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 346,
3
+ "best_metric": 0.9295058139534884,
4
+ "best_model_checkpoint": "./results_google_vit-large-patch16-224-in21k/checkpoint-346",
5
+ "epoch": 2.0,
6
+ "eval_steps": 500,
7
+ "global_step": 346,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.28901734104046245,
14
+ "grad_norm": 5.746753215789795,
15
+ "learning_rate": 2.45e-05,
16
+ "loss": 1.5215,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.5780346820809249,
21
+ "grad_norm": 2.093648672103882,
22
+ "learning_rate": 4.9500000000000004e-05,
23
+ "loss": 0.4825,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.8670520231213873,
28
+ "grad_norm": 3.6535494327545166,
29
+ "learning_rate": 4.738805970149254e-05,
30
+ "loss": 0.3352,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 1.0,
35
+ "eval_accuracy": 0.8997093023255814,
36
+ "eval_f1": 0.9002075736375416,
37
+ "eval_loss": 0.3576313555240631,
38
+ "eval_precision": 0.905387724540619,
39
+ "eval_recall": 0.8997093023255814,
40
+ "eval_runtime": 36.7463,
41
+ "eval_samples_per_second": 37.446,
42
+ "eval_steps_per_second": 4.681,
43
+ "step": 173
44
+ },
45
+ {
46
+ "epoch": 1.1560693641618498,
47
+ "grad_norm": 0.14127115905284882,
48
+ "learning_rate": 4.47228144989339e-05,
49
+ "loss": 0.1697,
50
+ "step": 200
51
+ },
52
+ {
53
+ "epoch": 1.4450867052023122,
54
+ "grad_norm": 0.16555330157279968,
55
+ "learning_rate": 4.2057569296375274e-05,
56
+ "loss": 0.2157,
57
+ "step": 250
58
+ },
59
+ {
60
+ "epoch": 1.7341040462427746,
61
+ "grad_norm": 8.537938117980957,
62
+ "learning_rate": 3.9392324093816635e-05,
63
+ "loss": 0.1304,
64
+ "step": 300
65
+ },
66
+ {
67
+ "epoch": 2.0,
68
+ "eval_accuracy": 0.9295058139534884,
69
+ "eval_f1": 0.9299410884859952,
70
+ "eval_loss": 0.2657730281352997,
71
+ "eval_precision": 0.933292642170999,
72
+ "eval_recall": 0.9295058139534884,
73
+ "eval_runtime": 36.4406,
74
+ "eval_samples_per_second": 37.76,
75
+ "eval_steps_per_second": 4.72,
76
+ "step": 346
77
+ }
78
+ ],
79
+ "logging_steps": 50,
80
+ "max_steps": 1038,
81
+ "num_input_tokens_seen": 0,
82
+ "num_train_epochs": 6,
83
+ "save_steps": 500,
84
+ "stateful_callbacks": {
85
+ "TrainerControl": {
86
+ "args": {
87
+ "should_epoch_stop": false,
88
+ "should_evaluate": false,
89
+ "should_log": false,
90
+ "should_save": true,
91
+ "should_training_stop": false
92
+ },
93
+ "attributes": {}
94
+ }
95
+ },
96
+ "total_flos": 7.577224264394588e+17,
97
+ "train_batch_size": 8,
98
+ "trial_name": null,
99
+ "trial_params": null
100
+ }
checkpoint-346/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6c9c3f17910ac792342cf93f2277214bda780f04d3229697f0bcc6d445c55e
3
+ size 5432
checkpoint-519/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-519/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44212eeb777687fdcfe1fd1dec2bd3ad740139fca9dff07d872fb63ac25e11bd
3
+ size 1213294072
checkpoint-519/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e527aecf26aebb4670f4dbe318652294138db25c5eed18a57186d7248e6e1ed2
3
+ size 2426823082
checkpoint-519/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }
checkpoint-519/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95b6047bd8cc6f4cdf7c46dea47edb8e542435510070c6cd1e0a7d9ccf5fd7da
3
+ size 14244
checkpoint-519/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71bd1fa9715204c338ba6bcc2faef7c7b31ed26468ed0ead59bd8e738b1d85e2
3
+ size 988
checkpoint-519/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7450cf42f4bceacb74b43e045d5a0d345372a37ae04be60cdca89d44a5b9d808
3
+ size 1064
checkpoint-519/trainer_state.json ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 519,
3
+ "best_metric": 0.9491279069767442,
4
+ "best_model_checkpoint": "./results_google_vit-large-patch16-224-in21k/checkpoint-519",
5
+ "epoch": 3.0,
6
+ "eval_steps": 500,
7
+ "global_step": 519,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.28901734104046245,
14
+ "grad_norm": 5.746753215789795,
15
+ "learning_rate": 2.45e-05,
16
+ "loss": 1.5215,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.5780346820809249,
21
+ "grad_norm": 2.093648672103882,
22
+ "learning_rate": 4.9500000000000004e-05,
23
+ "loss": 0.4825,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.8670520231213873,
28
+ "grad_norm": 3.6535494327545166,
29
+ "learning_rate": 4.738805970149254e-05,
30
+ "loss": 0.3352,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 1.0,
35
+ "eval_accuracy": 0.8997093023255814,
36
+ "eval_f1": 0.9002075736375416,
37
+ "eval_loss": 0.3576313555240631,
38
+ "eval_precision": 0.905387724540619,
39
+ "eval_recall": 0.8997093023255814,
40
+ "eval_runtime": 36.7463,
41
+ "eval_samples_per_second": 37.446,
42
+ "eval_steps_per_second": 4.681,
43
+ "step": 173
44
+ },
45
+ {
46
+ "epoch": 1.1560693641618498,
47
+ "grad_norm": 0.14127115905284882,
48
+ "learning_rate": 4.47228144989339e-05,
49
+ "loss": 0.1697,
50
+ "step": 200
51
+ },
52
+ {
53
+ "epoch": 1.4450867052023122,
54
+ "grad_norm": 0.16555330157279968,
55
+ "learning_rate": 4.2057569296375274e-05,
56
+ "loss": 0.2157,
57
+ "step": 250
58
+ },
59
+ {
60
+ "epoch": 1.7341040462427746,
61
+ "grad_norm": 8.537938117980957,
62
+ "learning_rate": 3.9392324093816635e-05,
63
+ "loss": 0.1304,
64
+ "step": 300
65
+ },
66
+ {
67
+ "epoch": 2.0,
68
+ "eval_accuracy": 0.9295058139534884,
69
+ "eval_f1": 0.9299410884859952,
70
+ "eval_loss": 0.2657730281352997,
71
+ "eval_precision": 0.933292642170999,
72
+ "eval_recall": 0.9295058139534884,
73
+ "eval_runtime": 36.4406,
74
+ "eval_samples_per_second": 37.76,
75
+ "eval_steps_per_second": 4.72,
76
+ "step": 346
77
+ },
78
+ {
79
+ "epoch": 2.023121387283237,
80
+ "grad_norm": 0.20084255933761597,
81
+ "learning_rate": 3.6727078891258e-05,
82
+ "loss": 0.1167,
83
+ "step": 350
84
+ },
85
+ {
86
+ "epoch": 2.3121387283236996,
87
+ "grad_norm": 0.020546281710267067,
88
+ "learning_rate": 3.4061833688699365e-05,
89
+ "loss": 0.0301,
90
+ "step": 400
91
+ },
92
+ {
93
+ "epoch": 2.601156069364162,
94
+ "grad_norm": 0.38922008872032166,
95
+ "learning_rate": 3.139658848614073e-05,
96
+ "loss": 0.0882,
97
+ "step": 450
98
+ },
99
+ {
100
+ "epoch": 2.8901734104046244,
101
+ "grad_norm": 0.017280207946896553,
102
+ "learning_rate": 2.8731343283582092e-05,
103
+ "loss": 0.075,
104
+ "step": 500
105
+ },
106
+ {
107
+ "epoch": 3.0,
108
+ "eval_accuracy": 0.9491279069767442,
109
+ "eval_f1": 0.9493183237288035,
110
+ "eval_loss": 0.23121964931488037,
111
+ "eval_precision": 0.9533860558309619,
112
+ "eval_recall": 0.9491279069767442,
113
+ "eval_runtime": 36.6312,
114
+ "eval_samples_per_second": 37.564,
115
+ "eval_steps_per_second": 4.695,
116
+ "step": 519
117
+ }
118
+ ],
119
+ "logging_steps": 50,
120
+ "max_steps": 1038,
121
+ "num_input_tokens_seen": 0,
122
+ "num_train_epochs": 6,
123
+ "save_steps": 500,
124
+ "stateful_callbacks": {
125
+ "TrainerControl": {
126
+ "args": {
127
+ "should_epoch_stop": false,
128
+ "should_evaluate": false,
129
+ "should_log": false,
130
+ "should_save": true,
131
+ "should_training_stop": false
132
+ },
133
+ "attributes": {}
134
+ }
135
+ },
136
+ "total_flos": 1.1365836396591882e+18,
137
+ "train_batch_size": 8,
138
+ "trial_name": null,
139
+ "trial_params": null
140
+ }
checkpoint-519/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6c9c3f17910ac792342cf93f2277214bda780f04d3229697f0bcc6d445c55e
3
+ size 5432
checkpoint-692/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-692/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:718a7034f83971c4cfc0da9674d4cf2d4f83fc42c49138f91da30ffab364dd42
3
+ size 1213294072
checkpoint-692/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d956dcce572eb9310c74fc501c364a610e0efdd7be49948a06cbe7c9bb2e042d
3
+ size 2426823082
checkpoint-692/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }
checkpoint-692/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2768285b45b2a0c05f6f50bbb8c0287fca6f62a8cde6d1b1f02151ac72ee8dc
3
+ size 14244
checkpoint-692/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee4ee5b4bf64535e1c4e7b8cb1191f8f02942b7b81aa4a353dd0948ce043d858
3
+ size 988
checkpoint-692/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48fb7502cf263297f4d041ee8dd5df349c2bf69e1f67058a2d6f741dd32c67f1
3
+ size 1064
checkpoint-692/trainer_state.json ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 519,
3
+ "best_metric": 0.9491279069767442,
4
+ "best_model_checkpoint": "./results_google_vit-large-patch16-224-in21k/checkpoint-519",
5
+ "epoch": 4.0,
6
+ "eval_steps": 500,
7
+ "global_step": 692,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.28901734104046245,
14
+ "grad_norm": 5.746753215789795,
15
+ "learning_rate": 2.45e-05,
16
+ "loss": 1.5215,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.5780346820809249,
21
+ "grad_norm": 2.093648672103882,
22
+ "learning_rate": 4.9500000000000004e-05,
23
+ "loss": 0.4825,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.8670520231213873,
28
+ "grad_norm": 3.6535494327545166,
29
+ "learning_rate": 4.738805970149254e-05,
30
+ "loss": 0.3352,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 1.0,
35
+ "eval_accuracy": 0.8997093023255814,
36
+ "eval_f1": 0.9002075736375416,
37
+ "eval_loss": 0.3576313555240631,
38
+ "eval_precision": 0.905387724540619,
39
+ "eval_recall": 0.8997093023255814,
40
+ "eval_runtime": 36.7463,
41
+ "eval_samples_per_second": 37.446,
42
+ "eval_steps_per_second": 4.681,
43
+ "step": 173
44
+ },
45
+ {
46
+ "epoch": 1.1560693641618498,
47
+ "grad_norm": 0.14127115905284882,
48
+ "learning_rate": 4.47228144989339e-05,
49
+ "loss": 0.1697,
50
+ "step": 200
51
+ },
52
+ {
53
+ "epoch": 1.4450867052023122,
54
+ "grad_norm": 0.16555330157279968,
55
+ "learning_rate": 4.2057569296375274e-05,
56
+ "loss": 0.2157,
57
+ "step": 250
58
+ },
59
+ {
60
+ "epoch": 1.7341040462427746,
61
+ "grad_norm": 8.537938117980957,
62
+ "learning_rate": 3.9392324093816635e-05,
63
+ "loss": 0.1304,
64
+ "step": 300
65
+ },
66
+ {
67
+ "epoch": 2.0,
68
+ "eval_accuracy": 0.9295058139534884,
69
+ "eval_f1": 0.9299410884859952,
70
+ "eval_loss": 0.2657730281352997,
71
+ "eval_precision": 0.933292642170999,
72
+ "eval_recall": 0.9295058139534884,
73
+ "eval_runtime": 36.4406,
74
+ "eval_samples_per_second": 37.76,
75
+ "eval_steps_per_second": 4.72,
76
+ "step": 346
77
+ },
78
+ {
79
+ "epoch": 2.023121387283237,
80
+ "grad_norm": 0.20084255933761597,
81
+ "learning_rate": 3.6727078891258e-05,
82
+ "loss": 0.1167,
83
+ "step": 350
84
+ },
85
+ {
86
+ "epoch": 2.3121387283236996,
87
+ "grad_norm": 0.020546281710267067,
88
+ "learning_rate": 3.4061833688699365e-05,
89
+ "loss": 0.0301,
90
+ "step": 400
91
+ },
92
+ {
93
+ "epoch": 2.601156069364162,
94
+ "grad_norm": 0.38922008872032166,
95
+ "learning_rate": 3.139658848614073e-05,
96
+ "loss": 0.0882,
97
+ "step": 450
98
+ },
99
+ {
100
+ "epoch": 2.8901734104046244,
101
+ "grad_norm": 0.017280207946896553,
102
+ "learning_rate": 2.8731343283582092e-05,
103
+ "loss": 0.075,
104
+ "step": 500
105
+ },
106
+ {
107
+ "epoch": 3.0,
108
+ "eval_accuracy": 0.9491279069767442,
109
+ "eval_f1": 0.9493183237288035,
110
+ "eval_loss": 0.23121964931488037,
111
+ "eval_precision": 0.9533860558309619,
112
+ "eval_recall": 0.9491279069767442,
113
+ "eval_runtime": 36.6312,
114
+ "eval_samples_per_second": 37.564,
115
+ "eval_steps_per_second": 4.695,
116
+ "step": 519
117
+ },
118
+ {
119
+ "epoch": 3.179190751445087,
120
+ "grad_norm": 0.01825469546020031,
121
+ "learning_rate": 2.6066098081023454e-05,
122
+ "loss": 0.0221,
123
+ "step": 550
124
+ },
125
+ {
126
+ "epoch": 3.468208092485549,
127
+ "grad_norm": 0.3856063485145569,
128
+ "learning_rate": 2.345415778251599e-05,
129
+ "loss": 0.0156,
130
+ "step": 600
131
+ },
132
+ {
133
+ "epoch": 3.7572254335260116,
134
+ "grad_norm": 0.024169214069843292,
135
+ "learning_rate": 2.0788912579957357e-05,
136
+ "loss": 0.0384,
137
+ "step": 650
138
+ },
139
+ {
140
+ "epoch": 4.0,
141
+ "eval_accuracy": 0.940406976744186,
142
+ "eval_f1": 0.9397796560818565,
143
+ "eval_loss": 0.25720012187957764,
144
+ "eval_precision": 0.9423321815760374,
145
+ "eval_recall": 0.940406976744186,
146
+ "eval_runtime": 36.5035,
147
+ "eval_samples_per_second": 37.695,
148
+ "eval_steps_per_second": 4.712,
149
+ "step": 692
150
+ }
151
+ ],
152
+ "logging_steps": 50,
153
+ "max_steps": 1038,
154
+ "num_input_tokens_seen": 0,
155
+ "num_train_epochs": 6,
156
+ "save_steps": 500,
157
+ "stateful_callbacks": {
158
+ "TrainerControl": {
159
+ "args": {
160
+ "should_epoch_stop": false,
161
+ "should_evaluate": false,
162
+ "should_log": false,
163
+ "should_save": true,
164
+ "should_training_stop": false
165
+ },
166
+ "attributes": {}
167
+ }
168
+ },
169
+ "total_flos": 1.5154448528789176e+18,
170
+ "train_batch_size": 8,
171
+ "trial_name": null,
172
+ "trial_params": null
173
+ }
checkpoint-692/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6c9c3f17910ac792342cf93f2277214bda780f04d3229697f0bcc6d445c55e
3
+ size 5432
checkpoint-865/config.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ViTForImageClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.0,
6
+ "encoder_stride": 16,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.0,
9
+ "hidden_size": 1024,
10
+ "id2label": {
11
+ "0": "Crescente_AZAN",
12
+ "1": "Crescente_HE",
13
+ "2": "Crescente_PAMS",
14
+ "3": "Crescente_PAS",
15
+ "4": "Crescente_PICRO",
16
+ "5": "Normal_AZAN",
17
+ "6": "Normal_HE",
18
+ "7": "Normal_PAMS",
19
+ "8": "Normal_PAS",
20
+ "9": "Normal_PICRO"
21
+ },
22
+ "image_size": 224,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 4096,
25
+ "label2id": {
26
+ "Crescente_AZAN": 0,
27
+ "Crescente_HE": 1,
28
+ "Crescente_PAMS": 2,
29
+ "Crescente_PAS": 3,
30
+ "Crescente_PICRO": 4,
31
+ "Normal_AZAN": 5,
32
+ "Normal_HE": 6,
33
+ "Normal_PAMS": 7,
34
+ "Normal_PAS": 8,
35
+ "Normal_PICRO": 9
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "model_type": "vit",
39
+ "num_attention_heads": 16,
40
+ "num_channels": 3,
41
+ "num_hidden_layers": 24,
42
+ "patch_size": 16,
43
+ "pooler_act": "tanh",
44
+ "pooler_output_size": 1024,
45
+ "problem_type": "single_label_classification",
46
+ "qkv_bias": true,
47
+ "torch_dtype": "float32",
48
+ "transformers_version": "4.53.1"
49
+ }
checkpoint-865/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46f4fde21629696f8b3abb99ce0ada867fa0e54a08510b4ab421e50916b02432
3
+ size 1213294072
checkpoint-865/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62639bec4251ebaf596b4b0171292a7b6aeae2ea9b12914ceaf968c7fee2b626
3
+ size 2426823082
checkpoint-865/preprocessor_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_rescale": true,
5
+ "do_resize": true,
6
+ "image_mean": [
7
+ 0.5,
8
+ 0.5,
9
+ 0.5
10
+ ],
11
+ "image_processor_type": "ViTImageProcessor",
12
+ "image_std": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "resample": 2,
18
+ "rescale_factor": 0.00392156862745098,
19
+ "size": {
20
+ "height": 224,
21
+ "width": 224
22
+ }
23
+ }