File size: 3,553 Bytes
c035918 49d8b99 c035918 0293479 c035918 0293479 c035918 0293479 c035918 5e06952 c035918 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
language: cs
license: cc-by-nc-sa-4.0
tags:
- Czech
- GEC
- GECCC dataset
pipeline_tag: text-generation
library_name: transformers
base_model: google/byt5-base
---
# Model Card for byt5-base-geccc-mate
The `byt5-base-geccc-mate` model is a sequence-to-sequence model performing
grammar error correction in Czech described in the paper
[Refining Czech GEC: Insights from a Multi-Experiment Approach](https://arxiv.org/abs/2506.22402).
It is a finetuned version of [byt5-base](https://huggingface.co/google/byt5-base) using
the MATE method and the [GECCC dataset](https://hdl.handle.net/11234/1-4861).
## Model Description
- **Developed by:** [Seznam.cz](https://seznam.cz) and [Charles University, MFF, ÚFAL](https://ufal.mff.cuni.cz/)
- **Language(s) (NLP):** Czech
- **Model type:** character-based encoder-decoder Transformer model
- **Finetuned from model:** `google/byt5-base`
- **Finetuned on:**
- first synthetic errors generated by the MATE method (see [the paper](https://arxiv.org/abs/2506.22402))
- then the [GECCC dataset](https://hdl.handle.net/11234/1-4861)
- **License:** CC BY-NC-SA 4.0
## Model Sources
- **Repository:** https://github.com/ufal/tsd2025-gec
- **Paper:** [Refining Czech GEC: Insights from a Multi-Experiment Approach](https://arxiv.org/abs/2506.22402)
- **Dataset:** [GECCC dataset](https://hdl.handle.net/11234/1-4861)
## Evaluation
<div align="center">
<img src="https://github.com/ufal/tsd2025-gec/blob/main/figures/bubble_chart.svg?raw=true" width="75%" alt="Performance bubblechart" />
</div>
| Model | Parameters | GECCC F-0.5 score | AKCES F-0.5 score |
|:------|-----------:|:-----------------:|:-----------------:|
| [byt5-small-geccc-mate](https://hf.co/ufal/byt5-small-geccc-mate) | 300M | 72.56 |
| [**byt5-base-geccc-mate**](https://hf.co/ufal/byt5-base-geccc-mate) | **582M** | **75.15** |
| [byt5-large-geccc-mate](https://hf.co/ufal/byt5-large-geccc-mate) | 1275M | 77.01 |
| [byt5-large-akces-mate](https://hf.co/ufal/byt5-large-akces-mate) | 1275M | | 84.40 |
| [transformer-base-geccc-mate](https://hf.co/ufal/transformer-base-geccc-mate) | 65M | 73.73 |
## Uses
The model can be directly used to process space-tokenized input Czech text and produce grammar-corrected Czech text.
## How to Get Started with the Model
Use the code below to get started with the model. Note that the input must be **space-tokenized**, i.e., every token (using the [UDPipe 1](https://ufal.mff.cuni.cz/udpipe/1) tokenizer [czech-pdt-ud-2.5-191206.udpipe](https://hdl.handle.net/11234/1-3131)) must be space-separated.
```python
tokenizer = transformers.AutoTokenizer.from_pretrained("ufal/byt5-base-geccc-mate")
model = transformers.AutoModelForSeq2SeqLM.from_pretrained("ufal/byt5-base-geccc-mate")
batch = tokenizer(["Sveřepý šakali zavile vyly na býlí mesýc ."], return_tensors="pt")
outputs = model.generate(batch.input_ids, max_length=256, num_beams=4)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
```
## BibTeX Citation
```
@InProceedings{10.1007/978-3-032-02551-7_7,
author="Pechman, Petr and Straka, Milan and Strakov{\'a}, Jana and N{\'a}plava, Jakub",
editor="Ek{\v{s}}tein, Kamil and Konop{\'i}k, Miloslav and Pra{\v{z}}{\'a}k, Ond{\v{r}}ej and P{\'a}rtl, Franti{\v{s}}ek",
title="Refining Czech GEC: Insights from a Multi-experiment Approach",
booktitle="Text, Speech, and Dialogue",
year="2026",
publisher="Springer Nature Switzerland",
address="Cham",
pages="64--76",
isbn="978-3-032-02551-7",
doi="10.1007/978-3-032-02551-7_7"
}
```
|