roberta-base-Disease-NER

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7496
• Precision: 0.5450
• Recall: 0.6759
• F1: 0.6035
• Accuracy: 0.8198

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	180	0.7775	0.3892	0.5483	0.4552	0.7676
No log	2.0	360	0.5731	0.4717	0.6003	0.5283	0.8152
0.8746	3.0	540	0.5629	0.4745	0.6515	0.5491	0.8164
0.8746	4.0	720	0.5848	0.4603	0.6744	0.5472	0.8106
0.8746	5.0	900	0.5489	0.5212	0.6686	0.5858	0.8239
0.4396	6.0	1080	0.5524	0.5123	0.6804	0.5845	0.8195
0.4396	7.0	1260	0.5550	0.5001	0.6842	0.5778	0.8174
0.4396	8.0	1440	0.5787	0.4982	0.6882	0.5780	0.8128
0.3302	9.0	1620	0.5824	0.5104	0.6939	0.5882	0.8154
0.3302	10.0	1800	0.5872	0.5295	0.6781	0.5947	0.8211
0.3302	11.0	1980	0.6047	0.5261	0.6867	0.5957	0.8210
0.2564	12.0	2160	0.6151	0.5357	0.6739	0.5969	0.8220
0.2564	13.0	2340	0.6560	0.5204	0.6784	0.5890	0.8172
0.204	14.0	2520	0.6866	0.5162	0.6919	0.5913	0.8155
0.204	15.0	2700	0.6994	0.5192	0.6887	0.5921	0.8145
0.204	16.0	2880	0.6904	0.5309	0.6764	0.5949	0.8199
0.1655	17.0	3060	0.7752	0.4925	0.6919	0.5754	0.8059
0.1655	18.0	3240	0.7464	0.5182	0.6832	0.5893	0.8152
0.1655	19.0	3420	0.7739	0.5242	0.6784	0.5914	0.8157
0.1335	20.0	3600	0.7496	0.5450	0.6759	0.6035	0.8198
0.1335	21.0	3780	0.7835	0.5296	0.6759	0.5939	0.8141
0.1335	22.0	3960	0.8174	0.5080	0.6869	0.5841	0.8092
0.1155	23.0	4140	0.8307	0.5336	0.6746	0.5959	0.8153
0.1155	24.0	4320	0.8457	0.5253	0.6832	0.5939	0.8126
0.0959	25.0	4500	0.8473	0.5250	0.6829	0.5936	0.8138
0.0959	26.0	4680	0.8971	0.5131	0.6837	0.5862	0.8069
0.0959	27.0	4860	0.8770	0.5229	0.6849	0.5930	0.8161
0.0814	28.0	5040	0.9317	0.5012	0.6894	0.5804	0.8083
0.0814	29.0	5220	0.9051	0.5288	0.6776	0.5940	0.8141
0.0814	30.0	5400	0.9387	0.5184	0.6839	0.5897	0.8106
0.0706	31.0	5580	0.9402	0.5261	0.6897	0.5969	0.8134
0.0706	32.0	5760	0.9603	0.5121	0.6839	0.5857	0.8104
0.0706	33.0	5940	0.9535	0.5255	0.6769	0.5917	0.8145
0.062	34.0	6120	0.9675	0.5250	0.6844	0.5942	0.8142
0.062	35.0	6300	0.9938	0.5249	0.6754	0.5907	0.8128
0.062	36.0	6480	0.9890	0.5222	0.6796	0.5906	0.8124
0.0544	37.0	6660	1.0106	0.5244	0.6794	0.5919	0.8135
0.0544	38.0	6840	1.0285	0.5230	0.6839	0.5928	0.8109
0.0489	39.0	7020	1.0253	0.5219	0.6809	0.5909	0.8137
0.0489	40.0	7200	1.0263	0.5229	0.6806	0.5914	0.8124
0.0489	41.0	7380	1.0511	0.5205	0.6849	0.5915	0.8113
0.0447	42.0	7560	1.0563	0.5145	0.6804	0.5859	0.8110
0.0447	43.0	7740	1.0521	0.5210	0.6814	0.5905	0.8128
0.0447	44.0	7920	1.0581	0.5220	0.6799	0.5906	0.8115
0.0411	45.0	8100	1.0597	0.5221	0.6816	0.5913	0.8127
0.0411	46.0	8280	1.0770	0.5216	0.6844	0.5920	0.8114
0.0411	47.0	8460	1.0689	0.5275	0.6847	0.5959	0.8128
0.039	48.0	8640	1.0665	0.5284	0.6821	0.5955	0.8135
0.039	49.0	8820	1.0715	0.5271	0.6829	0.5950	0.8128
0.0374	50.0	9000	1.0716	0.5273	0.6827	0.5950	0.8130

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2