 Preprocessing
 
 The texts are normalized using zenhan, segmented into words using Juman++, and tokenized using SentencePiece. Juman++ 2.0.0-rc3 was used for pretraining.
 
 The model was trained on 8 NVIDIA A100 GPUs.