deit-ena24_MD

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the ena24_MD dataset. It achieves the following results on the evaluation set:

• Loss: 1.8782
• Accuracy: 0.5459
• F1: 0.4851

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.1949	0.1259	100	2.2116	0.4688	0.3895
0.5955	0.2519	200	2.6496	0.4443	0.4240
0.6457	0.3778	300	1.8782	0.5459	0.4851
0.5871	0.5038	400	1.9099	0.6211	0.5460
0.5023	0.6297	500	1.9720	0.6328	0.5493
0.4641	0.7557	600	2.2744	0.5869	0.5344
0.7986	0.8816	700	2.3082	0.5947	0.5330
0.2118	1.0076	800	2.1929	0.6182	0.5418
0.3053	1.1335	900	2.2016	0.6533	0.5738
0.2434	1.2594	1000	2.4980	0.5908	0.5520
0.1593	1.3854	1100	2.3773	0.6279	0.5581
0.6254	1.5113	1200	2.4206	0.5898	0.5226
0.3366	1.6373	1300	2.8014	0.5762	0.5195
0.2613	1.7632	1400	2.6497	0.6172	0.5472
0.1136	1.8892	1500	2.4213	0.6074	0.5433
0.1071	2.0151	1600	2.5978	0.6436	0.5991
0.087	2.1411	1700	2.6259	0.6562	0.6066
0.0358	2.2670	1800	3.1009	0.5850	0.5267
0.1166	2.3929	1900	2.7449	0.6416	0.5866
0.0119	2.5189	2000	3.0153	0.625	0.5515
0.0563	2.6448	2100	2.8322	0.6387	0.5822
0.0241	2.7708	2200	2.8272	0.6318	0.5833
0.0844	2.8967	2300	2.8638	0.6416	0.5756
0.1294	3.0227	2400	2.8058	0.6484	0.5936
0.0602	3.1486	2500	2.9597	0.6279	0.5449
0.0445	3.2746	2600	2.7733	0.6602	0.5929
0.0528	3.4005	2700	2.5498	0.6826	0.6305
0.0323	3.5264	2800	2.7229	0.6719	0.6138
0.007	3.6524	2900	2.9141	0.6650	0.6025
0.0012	3.7783	3000	3.1029	0.6533	0.5968
0.0005	3.9043	3100	2.8776	0.6221	0.5639
0.0056	4.0302	3200	3.0144	0.6455	0.5882
0.0033	4.1562	3300	3.0141	0.6270	0.5702
0.0998	4.2821	3400	2.8958	0.6553	0.6073
0.0136	4.4081	3500	3.0230	0.6641	0.6053
0.0002	4.5340	3600	2.8966	0.6484	0.5750
0.0024	4.6599	3700	2.8534	0.6650	0.6010
0.0022	4.7859	3800	2.6686	0.6748	0.6247
0.0408	4.9118	3900	2.6715	0.6904	0.6327
0.1765	5.0378	4000	2.8339	0.6621	0.5940
0.0001	5.1637	4100	2.8891	0.6738	0.6109
0.0008	5.2897	4200	2.8508	0.6797	0.6183
0.0006	5.4156	4300	2.8630	0.6787	0.6190
0.0553	5.5416	4400	2.8210	0.6836	0.6239
0.0	5.6675	4500	2.7912	0.6865	0.6291
0.0	5.7935	4600	2.8061	0.6797	0.6237
0.0085	5.9194	4700	2.6576	0.6904	0.6332
0.0001	6.0453	4800	2.7866	0.6748	0.6211
0.0	6.1713	4900	2.8083	0.6738	0.6197
0.0	6.2972	5000	2.8078	0.6777	0.6251
0.0	6.4232	5100	2.7978	0.6797	0.6265
0.0	6.5491	5200	2.7378	0.6963	0.6421
0.0	6.6751	5300	2.7390	0.6973	0.6428
0.0	6.8010	5400	2.7374	0.6963	0.6417
0.0	6.9270	5500	2.7374	0.6963	0.6415

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1