|
--- |
|
library_name: transformers |
|
language: |
|
- ru |
|
metrics: |
|
- rouge |
|
base_model: |
|
- ai-forever/ruT5-base |
|
pipeline_tag: summarization |
|
--- |
|
|
|
# ruT5-base_headline_generation |
|
|
|
## Model Details |
|
|
|
T5 Base for news headline generation (Russian). The model is finetuned for best performance on short news texts (128 words or less), but it has decent metrics on longer articles as well. The model generates abstractive headlines that on average include 6-11 words. |
|
|
|
Base Model: [ai-forever/ruT5-base](https://huggingface.co/ai-forever/ruT5-base) |
|
|
|
## Training Details |
|
|
|
Training Data: 247 000 [news articles](https://www.dropbox.com/scl/fi/yq4ze46qm1aekku4u9y5w/ru_all_split.tar.gz?rlkey=wuzg9jcx0n15sten10k469pki&e=1) in Russian |
|
|
|
Training Procedure: 6 epochs, all details and hyperparameters in [this Google Colab notebook](https://colab.research.google.com/drive/1VVXufCGosV2qlW-UwtaPQna7hBHHmj8_?usp=sharing) |
|
|
|
## Testing Metrics |
|
|
|
- Rouge1: 40.24 |
|
- Rouge2: 23.05 |
|
- RougeL: 37.57 |
|
|
|
## How to Use |
|
|
|
```python |
|
from transformers import AutoTokenizer, T5ForConditionalGeneration |
|
|
|
model_name = "wanderer-msk/ruT5-base_headline_generation" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = T5ForConditionalGeneration.from_pretrained(model_name) |
|
|
|
news_text = """Земляне продолжают осваивать Марс. |
|
Колонисты уже посадили на красной планете 42 яблони.""" |
|
|
|
model_input = tokenizer( |
|
news_text, |
|
truncation=True, |
|
max_length=1024, |
|
return_tensors="pt" |
|
) |
|
model_output = model.generate(model_input["input_ids"]) |
|
news_headline = tokenizer.decode( |
|
model_output.squeeze(), |
|
skip_special_tokens=True |
|
) |
|
|
|
print(news_headline) |
|
``` |
