swin-CEMEDE

This model is a fine-tuned version of microsoft/swin-base-simmim-window6-192 on the cemede dataset. It achieves the following results on the evaluation set:

• Loss: 1.0793
• Accuracy: 0.7228
• F1: 0.6941

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.7728	0.0769	100	2.5932	0.2801	0.0798
2.0806	0.1538	200	2.6331	0.1826	0.0716
1.6491	0.2308	300	2.4971	0.2087	0.0878
1.3916	0.3077	400	2.4101	0.3400	0.1552
1.3267	0.3846	500	2.4311	0.2977	0.1745
1.2974	0.4615	600	1.8822	0.3913	0.2567
1.158	0.5385	700	2.2153	0.3799	0.2688
1.1604	0.6154	800	2.3287	0.4184	0.2898
0.9037	0.6923	900	1.8374	0.4736	0.3246
1.1772	0.7692	1000	1.8466	0.5178	0.4325
0.5862	0.8462	1100	2.0423	0.4779	0.4158
0.7642	0.9231	1200	1.7194	0.5307	0.4573
0.5552	1.0	1300	2.1011	0.5036	0.4280
0.5726	1.0769	1400	1.6062	0.5944	0.5339
0.5396	1.1538	1500	1.7786	0.5506	0.4801
0.598	1.2308	1600	1.1961	0.6543	0.5563
0.5893	1.3077	1700	1.9011	0.5925	0.4738
0.3518	1.3846	1800	1.5427	0.6063	0.5722
0.6365	1.4615	1900	1.5974	0.6163	0.5053
0.4551	1.5385	2000	1.1084	0.7071	0.6476
0.4621	1.6154	2100	1.9120	0.6082	0.5921
0.6878	1.6923	2200	1.5156	0.6415	0.5855
0.2327	1.7692	2300	1.4180	0.6581	0.6470
0.3159	1.8462	2400	1.3047	0.7009	0.6763
0.3304	1.9231	2500	1.6771	0.6643	0.6300
0.2685	2.0	2600	1.0793	0.7228	0.6941
0.0977	2.0769	2700	1.7003	0.6719	0.6534
0.3745	2.1538	2800	1.6521	0.6885	0.6616
0.2402	2.2308	2900	1.7074	0.6790	0.6616
0.3905	2.3077	3000	1.4096	0.7194	0.7111
0.0724	2.3846	3100	1.4934	0.7209	0.7247
0.2392	2.4615	3200	1.3840	0.7199	0.6977
0.33	2.5385	3300	1.3628	0.7256	0.7179
0.1502	2.6154	3400	1.2440	0.7542	0.7311
0.2107	2.6923	3500	1.2003	0.7708	0.7660
0.1965	2.7692	3600	1.7550	0.7071	0.7403

Framework versions

• Transformers 4.52.4
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1