transformers pytorch tensorboard mt5 text2text-generation summarization generated_from_trainer license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
metrics:
- rouge
tags:
- summarization
- generated_from_trainer
model-index:
- name: mt5-small-finetuned-31jan-4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mt5-small-finetuned-31jan-4

This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.5165
- Rouge1: 19.31
- Rouge2: 6.34
- Rougel: 19.06
- Rougelsum: 19.09

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 10
- eval_batch_size: 10
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|
| 6.4223        | 1.0   | 217  | 2.8162          | 13.51  | 3.46   | 13.13  | 13.26     |
| 3.4986        | 2.0   | 434  | 2.7158          | 15.95  | 4.09   | 15.66  | 15.77     |
| 3.2297        | 3.0   | 651  | 2.6552          | 16.82  | 4.3    | 16.4   | 16.52     |
| 3.0796        | 4.0   | 868  | 2.6526          | 17.99  | 5.02   | 17.6   | 17.79     |
| 2.969         | 5.0   | 1085 | 2.6005          | 18.05  | 5.22   | 17.78  | 17.79     |
| 2.8939        | 6.0   | 1302 | 2.5879          | 18.22  | 5.17   | 17.93  | 18.01     |
| 2.8147        | 7.0   | 1519 | 2.5569          | 18.25  | 5.56   | 18.03  | 18.14     |
| 2.7642        | 8.0   | 1736 | 2.5541          | 18.24  | 5.38   | 18.07  | 18.19     |
| 2.724         | 9.0   | 1953 | 2.5493          | 18.86  | 5.7    | 18.51  | 18.63     |
| 2.6962        | 10.0  | 2170 | 2.5320          | 19.12  | 5.72   | 18.93  | 19.01     |
| 2.6499        | 11.0  | 2387 | 2.5224          | 18.78  | 5.69   | 18.6   | 18.66     |
| 2.6242        | 12.0  | 2604 | 2.5272          | 19.23  | 5.82   | 18.96  | 18.99     |
| 2.6088        | 13.0  | 2821 | 2.5122          | 19.51  | 6.16   | 19.26  | 19.36     |
| 2.5976        | 14.0  | 3038 | 2.5218          | 19.06  | 6.23   | 18.82  | 18.87     |
| 2.5775        | 15.0  | 3255 | 2.5165          | 19.31  | 6.34   | 19.06  | 19.09     |


### Framework versions

- Transformers 4.26.0
- Pytorch 1.13.1+cu116
- Datasets 2.9.0
- Tokenizers 0.13.2
