my_awesome_billsum_model

This model is a fine-tuned version of t5-base on the billsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.6638
• Rouge1: 0.2033
• Rouge2: 0.1149
• Rougel: 0.1762
• Rougelsum: 0.1759
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	248	1.9584	0.1999	0.1073	0.1716	0.1717	19.0
No log	2.0	496	1.8621	0.195	0.1045	0.1685	0.1682	19.0
2.2512	3.0	744	1.8095	0.1973	0.1109	0.1728	0.1727	19.0
2.2512	4.0	992	1.7797	0.1989	0.1102	0.1724	0.1724	19.0
1.8144	5.0	1240	1.7505	0.1997	0.112	0.1735	0.1736	19.0
1.8144	6.0	1488	1.7308	0.2003	0.1134	0.1746	0.1744	19.0
1.6898	7.0	1736	1.7145	0.199	0.1114	0.1732	0.173	19.0
1.6898	8.0	1984	1.7083	0.1977	0.1106	0.1718	0.1716	19.0
1.5997	9.0	2232	1.6983	0.2014	0.1127	0.175	0.175	19.0
1.5997	10.0	2480	1.6923	0.2014	0.1153	0.1754	0.1753	19.0
1.5403	11.0	2728	1.6826	0.2009	0.1134	0.1752	0.1751	19.0
1.5403	12.0	2976	1.6768	0.2003	0.1125	0.1745	0.1744	19.0
1.491	13.0	3224	1.6722	0.2016	0.1146	0.1756	0.1755	19.0
1.491	14.0	3472	1.6750	0.2039	0.1164	0.1773	0.177	19.0
1.4496	15.0	3720	1.6679	0.2023	0.1147	0.1765	0.1763	19.0
1.4496	16.0	3968	1.6677	0.2032	0.1148	0.177	0.1768	19.0
1.4241	17.0	4216	1.6640	0.2021	0.1135	0.1752	0.175	19.0
1.4241	18.0	4464	1.6645	0.2027	0.1155	0.1766	0.1764	19.0
1.4025	19.0	4712	1.6632	0.2028	0.1149	0.1761	0.1757	19.0
1.4025	20.0	4960	1.6638	0.2033	0.1149	0.1762	0.1759	19.0

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1
• Datasets 2.14.1
• Tokenizers 0.13.3