bart_sum_samsum

This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5320
• Gen Len: 59.9242
• Rouge Score: {'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101}
• Bleu Score: {'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657}
• Bleurt Score: -0.4998
• Bert Score: [0.8805868625640869, 0.9189654588699341, 0.899208664894104]

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Gen Len	Rouge Score	Bleu Score	Bleurt Score	Bert Score
1.9517	1.0	921	1.8653	59.8374	{'rouge1': 0.38519198024299967, 'rouge2': 0.18637611248242514, 'rougeL': 0.29114807190727665, 'rougeLsum': 0.35950287045215523}	{'bleu': 0.10947202918144075, 'precisions': [0.2891732184886574, 0.1408997955010225, 0.07921257375593964, 0.04449898623412656], 'brevity_penalty': 1.0, 'length_ratio': 2.1406442622072146, 'translation_length': 39938, 'reference_length': 18657}	-0.5574	[0.881794273853302, 0.914982795715332, 0.897921621799469]
1.4162	2.0	1842	2.1673	60.6736	{'rouge1': 0.3824027985681461, 'rouge2': 0.17720440481192257, 'rougeL': 0.27951993033831063, 'rougeLsum': 0.3523751309023303}	{'bleu': 0.10292900287115767, 'precisions': [0.29144708090182264, 0.13358367689924108, 0.07251160668759896, 0.03975854026615448], 'brevity_penalty': 1.0, 'length_ratio': 2.084954708688428, 'translation_length': 38899, 'reference_length': 18657}	-0.7567	[0.873441755771637, 0.9113098978996277, 0.8918185234069824]
0.9763	3.0	2763	1.8854	59.8851	{'rouge1': 0.3925367542901428, 'rouge2': 0.19030742072418566, 'rougeL': 0.29557020575264703, 'rougeLsum': 0.36302164503856826}	{'bleu': 0.11050318220968344, 'precisions': [0.29364664926022627, 0.14059446150722135, 0.0786956634438425, 0.04589391170784672], 'brevity_penalty': 1.0, 'length_ratio': 2.1554912365332046, 'translation_length': 40215, 'reference_length': 18657}	-0.5280	[0.880211353302002, 0.9188302755355835, 0.8989349007606506]
0.5749	4.0	3684	2.1209	59.8313	{'rouge1': 0.39413787163188574, 'rouge2': 0.18797763014604468, 'rougeL': 0.29824353058090336, 'rougeLsum': 0.36387927887558746}	{'bleu': 0.10944201950995913, 'precisions': [0.2954957640803955, 0.1391474146019831, 0.07730156674867279, 0.045135857343175385], 'brevity_penalty': 1.0, 'length_ratio': 2.1574208072037306, 'translation_length': 40251, 'reference_length': 18657}	-0.5075	[0.8815322518348694, 0.9193716049194336, 0.89988774061203]
0.2765	5.0	4605	2.5320	59.9242	{'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101}	{'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657}	-0.4998	[0.8805868625640869, 0.9189654588699341, 0.899208664894104]

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.10.0
• Tokenizers 0.13.3