deit-CEMEDE

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the cemede dataset. It achieves the following results on the evaluation set:

• Loss: 0.8585
• Accuracy: 0.7884
• F1: 0.7973

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.5433	0.0769	100	1.4163	0.6339	0.4316
0.8153	0.1538	200	1.2107	0.7014	0.6080
0.4572	0.2308	300	0.9056	0.7513	0.6750
0.5055	0.3077	400	1.2329	0.6572	0.5795
0.5404	0.3846	500	1.1346	0.7156	0.6567
0.3872	0.4615	600	1.0172	0.7689	0.6869
0.2404	0.5385	700	1.1939	0.7299	0.7135
0.3426	0.6154	800	1.3790	0.7123	0.7013
0.3455	0.6923	900	1.2071	0.7223	0.6945
0.3843	0.7692	1000	2.4214	0.5982	0.6507
0.1851	0.8462	1100	0.9815	0.7618	0.7713
0.1783	0.9231	1200	0.8585	0.7884	0.7973
0.2812	1.0	1300	1.4161	0.7394	0.7475
0.0788	1.0769	1400	1.0477	0.7770	0.7855
0.1853	1.1538	1500	1.0843	0.7789	0.7907
0.0463	1.2308	1600	1.1819	0.7855	0.7422
0.1846	1.3077	1700	0.9227	0.7936	0.7835
0.0886	1.3846	1800	1.3368	0.7556	0.7603
0.1971	1.4615	1900	1.3276	0.7527	0.7413
0.2069	1.5385	2000	1.3338	0.7727	0.7805
0.1479	1.6154	2100	1.2606	0.7718	0.8021
0.1076	1.6923	2200	0.9955	0.8008	0.8183

Framework versions

• Transformers 4.52.4
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1