mt5-large-finetuned-norwegian

This model is a fine-tuned version of google/mt5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.8439
• Rouge1: 20.3786
• Rouge2: 9.3936
• Rougel: 17.7493
• Rougelsum: 18.6037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
9.2094	1.0	18	3.7149	16.3818	4.3429	13.0076	13.4039
4.1851	2.0	36	3.4349	18.7804	6.4406	15.4037	17.0282
3.4627	3.0	54	3.1672	19.8826	9.1029	16.2944	17.7482
3.0208	4.0	72	2.9852	20.9577	10.0017	16.8634	18.7444
2.7108	5.0	90	2.8552	21.0429	10.7747	18.0074	19.1268
2.6392	6.0	108	2.7793	23.4492	11.9814	20.2148	21.1514
2.4402	7.0	126	2.7472	21.8771	11.9814	18.9203	19.9279
2.4144	8.0	144	2.7253	21.0632	10.8657	19.1266	19.6957

Framework versions

• Transformers 4.51.3
• Pytorch 2.5.1+cu124
• Datasets 3.5.0
• Tokenizers 0.21.0