gmra_model_distilbert-base-uncased-distilled-squad_07112024T110436

This model is a fine-tuned version of distilbert-base-uncased-distilled-squad on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3023
• Accuracy: 94.1125
• F1: 0.9587

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.9982	142	0.3683	88.0492	0.7519
No log	1.9965	284	0.2634	91.5641	0.9238
No log	2.9947	426	0.2386	92.8822	0.9432
0.3507	4.0	569	0.2321	93.9367	0.9579
0.3507	4.9982	711	0.2897	93.4095	0.9536
0.3507	5.9965	853	0.2745	94.2882	0.9606
0.3507	6.9947	995	0.2892	94.3761	0.9616
0.0379	8.0	1138	0.3055	94.0246	0.9579
0.0379	8.9982	1280	0.3144	93.7610	0.9562
0.0379	9.9824	1420	0.3023	94.1125	0.9587

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1