transformers tensorboard safetensors t5 text2text-generation generated_from_trainer base_model:google-t5/t5-small base_model:finetune:google-t5/t5-small license:apache-2.0 autotrain_compatible text-generation-inference endpoints_compatible region:us
---
base_model: google-t5/t5-small
library_name: transformers
license: apache-2.0
metrics:
- rouge
tags:
- generated_from_trainer
model-index:
- name: my_summarization_model
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# my_summarization_model

This model is a fine-tuned version of [google-t5/t5-small](https://huggingface.co/google-t5/t5-small) on an billsum dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2280
- Rouge1: 0.4067
- Rouge2: 0.1832
- Rougel: 0.2719
- Rougelsum: 0.2717
- Gen Len: 126.8427

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 25
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len  |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:--------:|
| No log        | 1.0   | 124  | 2.5595          | 0.3549 | 0.1384 | 0.2234 | 0.2233    | 123.0242 |
| No log        | 2.0   | 248  | 2.4759          | 0.3779 | 0.1517 | 0.2462 | 0.2461    | 124.0323 |
| No log        | 3.0   | 372  | 2.4305          | 0.3921 | 0.1647 | 0.2583 | 0.2582    | 126.379  |
| No log        | 4.0   | 496  | 2.3922          | 0.393  | 0.1666 | 0.2609 | 0.261     | 126.1089 |
| 2.651         | 5.0   | 620  | 2.3726          | 0.3956 | 0.1689 | 0.2637 | 0.2641    | 126.3831 |
| 2.651         | 6.0   | 744  | 2.3473          | 0.3985 | 0.1736 | 0.2666 | 0.2669    | 126.4153 |
| 2.651         | 7.0   | 868  | 2.3269          | 0.3991 | 0.1717 | 0.2651 | 0.2651    | 126.4315 |
| 2.651         | 8.0   | 992  | 2.3154          | 0.3964 | 0.1695 | 0.2648 | 0.2647    | 126.5161 |
| 2.4496        | 9.0   | 1116 | 2.3047          | 0.4022 | 0.1755 | 0.2695 | 0.2694    | 126.5726 |
| 2.4496        | 10.0  | 1240 | 2.2988          | 0.4021 | 0.1758 | 0.27   | 0.2699    | 126.5161 |
| 2.4496        | 11.0  | 1364 | 2.2797          | 0.4033 | 0.1779 | 0.2718 | 0.2716    | 126.5726 |
| 2.4496        | 12.0  | 1488 | 2.2765          | 0.4072 | 0.1804 | 0.2719 | 0.2718    | 126.4758 |
| 2.3631        | 13.0  | 1612 | 2.2661          | 0.4074 | 0.1797 | 0.2722 | 0.2723    | 126.6452 |
| 2.3631        | 14.0  | 1736 | 2.2585          | 0.4042 | 0.1769 | 0.27   | 0.2698    | 126.6089 |
| 2.3631        | 15.0  | 1860 | 2.2539          | 0.4066 | 0.1797 | 0.2721 | 0.2722    | 126.6613 |
| 2.3631        | 16.0  | 1984 | 2.2497          | 0.403  | 0.176  | 0.2696 | 0.2697    | 126.6371 |
| 2.3203        | 17.0  | 2108 | 2.2438          | 0.4038 | 0.1783 | 0.2706 | 0.2707    | 126.7339 |
| 2.3203        | 18.0  | 2232 | 2.2375          | 0.4034 | 0.1787 | 0.2691 | 0.2693    | 126.7903 |
| 2.3203        | 19.0  | 2356 | 2.2354          | 0.4016 | 0.1779 | 0.2676 | 0.2677    | 126.8427 |
| 2.3203        | 20.0  | 2480 | 2.2334          | 0.4041 | 0.1787 | 0.2697 | 0.2697    | 126.8952 |
| 2.285         | 21.0  | 2604 | 2.2315          | 0.4026 | 0.1797 | 0.2694 | 0.2693    | 126.7903 |
| 2.285         | 22.0  | 2728 | 2.2302          | 0.4044 | 0.1804 | 0.27   | 0.27      | 126.7903 |
| 2.285         | 23.0  | 2852 | 2.2284          | 0.4055 | 0.1827 | 0.2716 | 0.2714    | 126.7379 |
| 2.285         | 24.0  | 2976 | 2.2283          | 0.4061 | 0.1825 | 0.2716 | 0.2715    | 126.7903 |
| 2.2698        | 25.0  | 3100 | 2.2280          | 0.4067 | 0.1832 | 0.2719 | 0.2717    | 126.8427 |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.19.1
