guymorlan/factrel_deberta

results

This model is a fine-tuned version of MoritzLaurer/deberta-v3-large-zeroshot-v2.0 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2757
• Accuracy: 0.8113
• F1 Macro: 0.7264
• Precision Neutral: 0.8559
• Recall Neutral: 0.8920
• F1 Neutral: 0.8736
• Support Neutral: 213
• Precision Entailment: 0.7436
• Recall Entailment: 0.5472
• F1 Entailment: 0.6304
• Support Entailment: 53
• Precision Contradiction: 0.6341
• Recall Contradiction: 0.7222
• F1 Contradiction: 0.6753
• Support Contradiction: 36

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro	Precision Neutral	Recall Neutral	F1 Neutral	Support Neutral	Precision Entailment	Recall Entailment	F1 Entailment	Support Entailment	Precision Contradiction	Recall Contradiction	F1 Contradiction	Support Contradiction
0.0311	1.0	383	0.2757	0.8113	0.7264	0.8559	0.8920	0.8736	213	0.7436	0.5472	0.6304	53	0.6341	0.7222	0.6753	36
0.0179	2.0	766	0.3661	0.7781	0.6858	0.8387	0.8545	0.8465	213	0.6226	0.6226	0.6226	53	0.625	0.5556	0.5882	36
0.003	3.0	1149	0.4953	0.8013	0.7063	0.8341	0.8967	0.8643	213	0.7045	0.5849	0.6392	53	0.6897	0.5556	0.6154	36
0.0008	4.0	1532	0.5224	0.7980	0.6984	0.8341	0.8967	0.8643	213	0.6977	0.5660	0.625	53	0.6667	0.5556	0.6061	36

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.1
• Tokenizers 0.21.1