bert-base-multilingual-cased-finetuned-conceptNet-te

This model is a fine-tuned version of bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0456

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	45	1.3708
No log	2.0	90	1.2736
No log	3.0	135	1.1219
No log	4.0	180	1.3133
1.3974	5.0	225	1.2969
1.3974	6.0	270	1.2416
1.3974	7.0	315	1.2252
1.3974	8.0	360	1.1990
1.0679	9.0	405	1.0824
1.0679	10.0	450	1.0680
1.0679	11.0	495	1.0228
1.0679	12.0	540	1.1910
1.0679	13.0	585	1.0335
0.9271	14.0	630	1.0809
0.9271	15.0	675	1.0019
0.9271	16.0	720	1.0216
0.9271	17.0	765	1.1762
0.8418	18.0	810	0.9434
0.8418	19.0	855	1.0431
0.8418	20.0	900	1.0873

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.3.1
• Tokenizers 0.21.0