Health_MainSections_PegasusLargeModel

This model is a fine-tuned version of google/pegasus-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.2414
• Rouge1: 50.1795
• Rouge2: 17.4664
• Rougel: 33.5547
• Rougelsum: 45.6086
• Bertscore Precision: 79.0557
• Bertscore Recall: 82.3708
• Bertscore F1: 80.6737
• Bleu: 0.1302
• Gen Len: 230.3297

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bertscore Precision	Bertscore Recall	Bertscore F1	Bleu	Gen Len
6.6491	0.0835	100	6.2799	39.9618	11.2774	25.6909	36.1285	76.1843	80.1799	78.1239	0.0837	230.3297
6.2174	0.1671	200	5.9405	42.5055	13.4081	28.4622	38.6545	76.9801	80.932	78.9	0.1013	230.3297
6.0623	0.2506	300	5.7883	44.9098	14.7089	29.9801	40.7648	77.6093	81.3921	79.4496	0.1118	230.3297
5.8987	0.3342	400	5.6549	46.317	15.8065	30.9576	41.9584	77.7634	81.688	79.6712	0.1182	230.3297
5.7395	0.4177	500	5.5221	47.7104	16.4157	31.8171	43.3162	78.0166	81.8334	79.8733	0.1221	230.3297
5.7026	0.5013	600	5.4323	47.7147	16.492	32.171	43.3503	78.266	81.9538	80.0615	0.1225	230.3297
5.6432	0.5848	700	5.3587	49.1073	17.0141	32.8254	44.5003	78.7001	82.1587	80.3864	0.1264	230.3297
5.6131	0.6684	800	5.3146	49.7061	17.3206	33.0712	45.1323	78.8811	82.2757	80.5371	0.1290	230.3297
5.5168	0.7519	900	5.2820	50.0791	17.439	33.333	45.4496	79.031	82.3405	80.6464	0.1300	230.3297
5.5748	0.8355	1000	5.2569	50.176	17.4666	33.4584	45.398	79.0421	82.3676	80.6652	0.1301	230.3297
5.4357	0.9190	1100	5.2414	50.1795	17.4664	33.5547	45.6086	79.0557	82.3708	80.6737	0.1302	230.3297

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.2.1
• Tokenizers 0.19.1