README.md

metadata

library_name: transformers
license: apache-2.0
base_model: facebook/deit-base-patch16-224
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: deit-base-patch16-224-finetuned-stroke-binary
    results: []

deit-base-patch16-224-finetuned-stroke-binary

This model is a fine-tuned version of facebook/deit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1572
• Accuracy: 0.9412
• F1: 0.9407
• Precision: 0.9419
• Recall: 0.9412

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 48
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.7318	0.6202	100	0.7300	0.4509	0.3933	0.5863	0.4509
0.6751	1.2357	200	0.6745	0.5699	0.5670	0.5647	0.5699
0.6521	1.8558	300	0.6379	0.6431	0.5742	0.6402	0.6431
0.5941	2.4713	400	0.5868	0.7010	0.6588	0.7256	0.7010
0.5435	3.0868	500	0.5232	0.7445	0.7133	0.7816	0.7445
0.4554	3.7070	600	0.4618	0.7820	0.7602	0.8189	0.7820
0.3992	4.3225	700	0.3778	0.8399	0.8327	0.8519	0.8399
0.3563	4.9426	800	0.3372	0.8494	0.8434	0.8596	0.8494
0.3286	5.5581	900	0.2941	0.8810	0.8785	0.8846	0.8810
0.2749	6.1736	1000	0.2696	0.8874	0.8854	0.8895	0.8874
0.2687	6.7938	1100	0.2890	0.8788	0.8744	0.8901	0.8788
0.26	7.4093	1200	0.2636	0.8901	0.8868	0.8988	0.8901
0.2624	8.0248	1300	0.2342	0.9082	0.9071	0.9092	0.9082
0.2853	8.6450	1400	0.2192	0.9132	0.9122	0.9143	0.9132
0.2153	9.2605	1500	0.2269	0.9104	0.9090	0.9130	0.9104
0.2288	9.8806	1600	0.2319	0.9082	0.9064	0.9124	0.9082
0.2233	10.4961	1700	0.2089	0.9177	0.9165	0.9201	0.9177
0.2006	11.1116	1800	0.2029	0.9209	0.9205	0.9207	0.9209
0.2059	11.7318	1900	0.1981	0.9199	0.9196	0.9198	0.9199
0.1993	12.3473	2000	0.2155	0.9168	0.9150	0.9220	0.9168
0.1925	12.9674	2100	0.1921	0.9258	0.9249	0.9274	0.9258
0.2067	13.5829	2200	0.1957	0.9267	0.9258	0.9286	0.9267
0.1856	14.1984	2300	0.1927	0.9272	0.9261	0.9297	0.9272
0.217	14.8186	2400	0.2155	0.9204	0.9188	0.9253	0.9204
0.1895	15.4341	2500	0.1782	0.9349	0.9343	0.9357	0.9349
0.2031	16.0496	2600	0.2666	0.8928	0.8888	0.9060	0.8928
0.1853	16.6698	2700	0.1845	0.9335	0.9331	0.9339	0.9335
0.1868	17.2853	2800	0.2151	0.9204	0.9185	0.9273	0.9204
0.1725	17.9054	2900	0.1789	0.9335	0.9330	0.9341	0.9335
0.1899	18.5209	3000	0.1704	0.9389	0.9384	0.9399	0.9389
0.1614	19.1364	3100	0.1761	0.9353	0.9348	0.9362	0.9353
0.166	19.7566	3200	0.1767	0.9362	0.9357	0.9372	0.9362
0.1783	20.3721	3300	0.1584	0.9403	0.9401	0.9403	0.9403
0.159	20.9922	3400	0.1572	0.9408	0.9403	0.9413	0.9408
0.1668	21.6078	3500	0.1652	0.9426	0.9419	0.9442	0.9426
0.1423	22.2233	3600	0.1601	0.9380	0.9376	0.9384	0.9380
0.1713	22.8434	3700	0.1572	0.9421	0.9417	0.9428	0.9421
0.1657	23.4589	3800	0.1579	0.9408	0.9403	0.9413	0.9408
0.1424	24.0744	3900	0.1689	0.9403	0.9397	0.9417	0.9403
0.169	24.6946	4000	0.1558	0.9444	0.9439	0.9451	0.9444
0.1439	25.3101	4100	0.1572	0.9412	0.9407	0.9419	0.9412

Framework versions

• Transformers 4.48.3
• Pytorch 2.6.0+cu124
• Datasets 3.4.0
• Tokenizers 0.21.0