PreTrain Electra/T5 for Korean from scratch
This project will be train the pretrained-model (Electra, T5, …) from Korean corpus called as a 모두의말뭉치.

2. Language
The model will be trained in Korean.

3. Model

Electra
T5


4. Datasets
모두의말뭉치 4

5. Training scripts
We can make use of example 21 to train the model.>
Addtionaly, we will be make the train codes origin from the transformers example

7. Desired project outcome
A pretrained weights. After, fine-tune both text classification and text summarization