readme: ---
  language: de
  tags:
  - summarization
  datasets:
  - mlsum
  ---
  
  # mT5-small fine-tuned on German MLSUM
  This model was finetuned for 3 epochs with a max_len (input) of 768 tokens and target_max_len of 192 tokens.  
  It was fine-tuned on all German articles present in the train split of the [MLSUM dataset](https://huggingface.co/datasets/mlsum) having less than 384 "words" after splitting on whitespace, which resulted in 80249 articles.  
  The exact expression to filter the dataset was the following:
  ```python
  dataset = dataset.filter(lambda e: len(e['text'].split()) < 384)
  ```
  
  ## Evaluation results
  The fine-tuned model was evaluated on 2000 random articles from the validation set.
  Mean [f1 ROUGE scores](https://github.com/pltrdy/rouge) were calculated for both the fine-tuned model and the lead-3 baseline (which simply produces the leading three sentences of the document) and are presented in the following table.
  
  | Model         | Rouge-1 | Rouge-2  | Rouge-L |
  | ------------- |:-------:| --------:| -------:|
  | mt5-small     | 0.399   | 0.318    | 0.392   |
  | lead-3        | 0.343   | 0.263    | 0.341   |