flan-t5-large-v1

This model is a fine-tuned version of google/flan-t5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1673
• Rouge1: 74.1287
• Rouge2: 66.4339
• Rougel: 72.8596
• Rougelsum: 73.8679
• Gen Len: 16.3241

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
12.4521	0.85	200	0.2565	72.2564	62.6872	70.741	72.0604	15.9467
0.2538	1.7	400	0.1877	72.8835	64.0804	71.5277	72.642	16.3582
0.1804	2.55	600	0.1715	73.307	64.5027	72.2345	73.098	16.1429
0.1516	3.4	800	0.1675	73.9648	65.6244	72.8421	73.8516	16.2026
0.1331	4.26	1000	0.1609	73.7382	65.6094	72.5124	73.5658	16.3198
0.1205	5.11	1200	0.1656	74.2505	66.5083	73.1059	74.0956	16.3795
0.1113	5.96	1400	0.1593	74.2997	66.2497	73.2158	74.1265	16.3326
0.1031	6.81	1600	0.1643	74.2861	66.3972	73.1252	74.0796	16.2729
0.0909	7.66	1800	0.1638	73.7071	65.61	72.4082	73.5071	16.3262
0.0876	8.51	2000	0.1667	74.1477	66.0628	72.9115	73.9177	16.3198
0.0911	9.36	2200	0.1673	74.1287	66.4339	72.8596	73.8679	16.3241

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3