 Corpus HerBERT was trained on six different corpora available for Polish language:  | Corpus | Tokens | Documents | | :------ | ------: | ------: | | [CCNet Middle](https://github.com/facebookresearch/cc_net) | 3243M  | 7.9M | | [CCNet Head](https://github.com/facebookresearch/cc_net) | 2641M  | 7.0M | | [National Corpus of Polish](http://nkjp.pl/index.php?page=14&lang=1)| 1357M  | 3.9M | | [Open Subtitles](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 1056M  | 1.1M  | [Wikipedia](https://dumps.wikimedia.org/) | 260M  | 1.4M | | [Wolne Lektury](https://wolnelektury.pl/) | 41M  | 5.5k |  