transformers pytorch safetensors t5 text2text-generation autotrain_compatible text-generation-inference endpoints_compatible region:us
---
{}
---
## T5-Small - Text summarization

### Task description

We are focusing on Abstractive Text Summarization. Briefly, the input of the task is a text paragraph and output is a summarization of the input which is similar to the input from its meaning. Compared to another approach (Extractive Text Summarization), Abstractive Text Summarization is outstanding in the output quality.

### Dataset 

The model was finetuned on [CNN/DailyMail](https://huggingface.co/datasets/cnn_dailymail). That is an English-language dataset containing just over 300k unique news articles as written by journalists at CNN and the Daily Mail. The current version supports both extractive and abstractive summarization, though the original version was created for machine reading and comprehension and abstractive question answering.

### Hyper-parameters

| Parameter     | Value                |
| ---------     |---------             |
| **No. Epoch**     | 3                    |
| **Learning rate** | 1e-5 (First two epochs), 5e-6 (Last epoch) |
| **Optimizer**    | AdamW                 |
| **Layers**        | Full                 |


### Evaluations

All of those metrics are from the evaluation of the finetuned model on the test-set of [CNN/DailyMail](https://huggingface.co/datasets/cnn_dailymail).

| Metrics | Recall | Precision | F1-Score |
| ------- | ------ | --------- | -------- |
| **Rouge 1** | 0.38 | 0.42 | 0.39 |
| **Rouge 2** | 0.16 | 0.18 | 0.17 |
| **Rouge L** | 0.27 | 0.3 | 0.27 |
| **Rouge L-Sum** | 0.27 | 0.3 | 0.27 |

---
license: apache-2.0
datasets:
- cnn_dailymail
language:
- en
metrics:
- rouge
library_name: transformers
pipeline_tag: summarization
---