transformers safetensors t5 text2text-generation summarization ru base_model:ai-forever/ruT5-base base_model:finetune:ai-forever/ruT5-base autotrain_compatible text-generation-inference endpoints_compatible region:us
---
base_model:
- ai-forever/ruT5-base
language:
- ru
library_name: transformers
metrics:
- rouge
pipeline_tag: summarization
---

# ruT5-base_headline_generation

## Model Details

T5 Base for news headline generation (Russian). The model is finetuned for best performance on short news texts (128 words or less), but it has decent metrics on longer articles as well. The model generates abstractive headlines that on average include 6-11 words.

Base Model: [ai-forever/ruT5-base](https://huggingface.co/ai-forever/ruT5-base)

## Training Details

Training Data: 247 000 [news articles](https://www.dropbox.com/scl/fi/yq4ze46qm1aekku4u9y5w/ru_all_split.tar.gz?rlkey=wuzg9jcx0n15sten10k469pki&e=1) in Russian

Training Procedure: 6 epochs, all details and hyperparameters in [this Google Colab notebook](https://colab.research.google.com/drive/1VVXufCGosV2qlW-UwtaPQna7hBHHmj8_?usp=sharing)

## Testing Metrics

- Rouge1: 40.24
- Rouge2: 23.05
- RougeL: 37.57

## How to Use

```python
from transformers import AutoTokenizer, T5ForConditionalGeneration

model_name = "wanderer-msk/ruT5-base_headline_generation"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

news_text = """Земляне продолжают осваивать Марс.
Колонисты уже посадили на красной планете 42 яблони."""

model_input = tokenizer(
    news_text,
    truncation=True,
    max_length=1024,
    return_tensors="pt"
)
model_output = model.generate(model_input["input_ids"])
news_headline = tokenizer.decode(
    model_output.squeeze(),
    skip_special_tokens=True
)

print(news_headline)
```