---
library_name: transformers
language:
- ru
metrics:
- rouge
base_model:
- ai-forever/ruT5-base
pipeline_tag: summarization
---

# ruT5-base_headline_generation

## Model Details

T5 Base for news headline generation (Russian). The model is finetuned for best performance on short news texts (128 words or less), but it has decent metrics on longer articles as well. The model generates abstractive headlines that on average include 6-11 words.

Base Model: [ai-forever/ruT5-base](https://huggingface.co/ai-forever/ruT5-base)

## Training Details

Training Data: 247 000 [news articles](https://www.dropbox.com/scl/fi/yq4ze46qm1aekku4u9y5w/ru_all_split.tar.gz?rlkey=wuzg9jcx0n15sten10k469pki&e=1) in Russian

Training Procedure: 6 epochs, all details and hyperparameters in [this Google Colab notebook](https://colab.research.google.com/drive/1VVXufCGosV2qlW-UwtaPQna7hBHHmj8_?usp=sharing)

## Testing Metrics

- Rouge1: 40.24
- Rouge2: 23.05
- RougeL: 37.57

## How to Use

```python
from transformers import AutoTokenizer, T5ForConditionalGeneration

model_name = "wanderer-msk/ruT5-base_headline_generation"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

news_text = """Земляне продолжают осваивать Марс.
Колонисты уже посадили на красной планете 42 яблони."""

model_input = tokenizer(
    news_text,
    truncation=True,
    max_length=1024,
    return_tensors="pt"
)
model_output = model.generate(model_input["input_ids"])
news_headline = tokenizer.decode(
    model_output.squeeze(),
    skip_special_tokens=True
)

print(news_headline)
```