xlnet-base-cased_fold_3_binary_v1

This model is a fine-tuned version of xlnet-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8649
• F1: 0.8044

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	289	0.4483	0.8000
0.4228	2.0	578	0.4264	0.8040
0.4228	3.0	867	0.5341	0.8056
0.2409	4.0	1156	0.9077	0.8103
0.2409	5.0	1445	1.1069	0.7889
0.1386	6.0	1734	1.0288	0.8093
0.0817	7.0	2023	1.2477	0.8049
0.0817	8.0	2312	1.5915	0.7872
0.0465	9.0	2601	1.5323	0.8035
0.0465	10.0	2890	1.4351	0.7989
0.0376	11.0	3179	1.4639	0.7916
0.0376	12.0	3468	1.6027	0.7956
0.0234	13.0	3757	1.7860	0.7931
0.0109	14.0	4046	1.8567	0.7934
0.0109	15.0	4335	1.8294	0.8053
0.0115	16.0	4624	1.7799	0.7971
0.0115	17.0	4913	1.5935	0.8000
0.0142	18.0	5202	1.8136	0.8066
0.0142	19.0	5491	1.7718	0.8063
0.0124	20.0	5780	1.8581	0.8053
0.0083	21.0	6069	1.8523	0.8056
0.0083	22.0	6358	1.8408	0.8035
0.0045	23.0	6647	1.8347	0.8040
0.0045	24.0	6936	1.8683	0.8067
0.0005	25.0	7225	1.8649	0.8044

Framework versions

• Transformers 4.21.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1