 Training data  Polyglot-Ko-1.3B was trained on 863 GB of Korean language data (1.2TB before processing), a large-scale dataset curated by [TUNiB](https://tunib.ai/). The data collection process has abided by South Korean laws. This dataset was collected for the purpose of training Polyglot-Ko models, so it will not be released for public use.    | Source                              |Size (GB) | Link                                  | |-------------------------------------|---------|------------------------------------------| | Korean blog posts                   | 682.3   | -                                        | | Korean news dataset                 | 87.0  | -                                        | | Modu corpus                         | 26.4  |corpus.korean.go.kr             | | Korean patent dataset               | 19.0  | -                                        | | Korean Q & A dataset                | 18.1  | -                                        | | KcBert dataset                      | 12.7  | github.com/Beomi/KcBERT          | | Korean fiction dataset              | 6.1   | -                                        | | Korean online comments              | 4.2   | -                                    | | Korean wikipedia                    | 1.4   | ko.wikipedia.org                 | | Clova call                          | < 1.0   | github.com/clovaai/ClovaCall     | | Naver sentiment movie corpus        | < 1.0   | github.com/e9t/nsmc              | | Korean hate speech dataset          | < 1.0   | -                                        | | Open subtitles                      | < 1.0  | opus.nlpl.eu/OpenSubtitles.php   | | AIHub various tasks datasets        | < 1.0   |aihub.or.kr                 | | Standard Korean language dictionary | < 1.0  | stdict.korean.go.kr/main/main.do |                              Furthermore, in order to avoid the model memorizing and generating personally identifiable information (PII) in the training data, we masked out the following sensitive information in the pre-processing stage:  * `<|acc|>` : bank account number * `<|rrn|>` : resident registration number * `<|tell|>` : phone number  