Kikia26/FineTunePubMedBertWithTensorflowKeras3

This model is a fine-tuned version of microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0981
• Validation Loss: 0.3764
• Train Precision: 0.6444
• Train Recall: 0.7342
• Train F1: 0.6864
• Train Accuracy: 0.9014
• Epoch: 12

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 200, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Precision	Train Recall	Train F1	Train Accuracy	Epoch
1.4820	0.8904	0.0	0.0	0.0	0.7808	0
0.8734	0.6681	0.6159	0.1793	0.2778	0.8274	1
0.6618	0.5098	0.6180	0.4641	0.5301	0.8673	2
0.4675	0.4214	0.6199	0.5781	0.5983	0.8841	3
0.3731	0.3833	0.5849	0.6540	0.6175	0.8910	4
0.2830	0.3550	0.6019	0.6730	0.6355	0.8958	5
0.2357	0.3555	0.6137	0.7004	0.6542	0.9025	6
0.2042	0.3500	0.6325	0.6646	0.6481	0.9004	7
0.1721	0.3511	0.5891	0.7046	0.6417	0.8964	8
0.1516	0.3692	0.6264	0.7004	0.6614	0.9017	9
0.1281	0.3477	0.6508	0.7194	0.6834	0.9046	10
0.1058	0.3701	0.6232	0.7257	0.6706	0.9012	11
0.0981	0.3764	0.6444	0.7342	0.6864	0.9014	12

Framework versions

• Transformers 4.35.2
• TensorFlow 2.14.0
• Datasets 2.15.0
• Tokenizers 0.15.0