phobert-v1-UIT-VSMEC-ep20

This model is a fine-tuned version of vinai/phobert-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.8803
• Micro F1: 60.2041
• Micro Precision: 60.2041
• Micro Recall: 60.2041
• Macro F1: 56.3187
• Macro Precision: 56.4421
• Macro Recall: 56.7758

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro Precision	Micro Recall	Macro F1	Macro Precision	Macro Recall
1.2873	1.0	87	1.1654	55.5394	55.5394	55.5394	46.8529	54.1029	49.0717
1.0336	2.0	174	1.2340	53.7901	53.7901	53.7901	48.8492	50.8904	48.0065
0.7343	3.0	261	1.2178	58.0175	58.0175	58.0175	52.1476	54.1567	52.5316
0.5816	4.0	348	1.3180	56.9971	56.9971	56.9971	50.9490	56.7674	51.8344
0.3814	5.0	435	1.4871	57.2886	57.2886	57.2886	52.7240	54.4479	53.4573
0.2262	6.0	522	1.7193	55.3936	55.3936	55.3936	50.9168	50.8368	54.1044
0.1993	7.0	609	1.6983	56.7055	56.7055	56.7055	52.4883	52.2002	53.6820
0.1824	8.0	696	1.7736	57.2886	57.2886	57.2886	52.1784	53.3327	53.1574
0.1039	9.0	783	2.1434	56.9971	56.9971	56.9971	53.6501	52.5002	56.3497
0.0972	10.0	870	2.0975	59.6210	59.6210	59.6210	56.0409	55.0219	57.6793
0.0576	11.0	957	2.3412	57.7259	57.7259	57.7259	53.2012	55.2565	52.3265
0.0324	12.0	1044	2.5451	58.6006	58.6006	58.6006	54.6677	53.3109	58.9052
0.0308	13.0	1131	2.5168	59.3294	59.3294	59.3294	55.3300	54.3472	58.2158
0.0335	14.0	1218	2.6728	59.7668	59.7668	59.7668	56.0250	55.0830	57.5383
0.0148	15.0	1305	2.7171	60.3499	60.3499	60.3499	55.9708	56.3880	56.4511
0.001	16.0	1392	2.7637	60.6414	60.6414	60.6414	56.7693	55.9442	58.0549
0.0014	17.0	1479	2.8592	60.0583	60.0583	60.0583	56.2963	56.2777	57.0295
0.0009	18.0	1566	2.8691	60.0583	60.0583	60.0583	56.1657	56.2744	56.5925
0.001	19.0	1653	2.8800	60.2041	60.2041	60.2041	56.2784	56.3492	56.7758
0.0008	19.7723	1720	2.8803	60.2041	60.2041	60.2041	56.3187	56.4421	56.7758

Framework versions

• Transformers 4.50.0
• Pytorch 2.6.0+cu124
• Datasets 2.15.0
• Tokenizers 0.21.1