readme: ---
  license: apache-2.0
  
  widget:
  - text: 横浜国立大学は日本の[MASK]奈川県にある。
  ---
  
  
  This is RoBERTa model pretrained on texts in the Japanese language.
  
  3.45GB wikipedia text
  
   trained 125M step
   
    use the BERT BPE tokenizer.
    
    If you want to fine-tune model. Please use 
  ```python
  
  
  from transformers import BertTokenizer, RoFormerModel
  BertTokenizer.from_pretrained('Roformer-base-japanese')
  RoFormerModel.from_pretrained('Roformer-base-japanese')
  ```
   
    
    The accuracy in JGLUE-marc-ja-v1.0 binary sentiment classification 95.12%
  
    
    
    
   Contribute by Yokohama Nationaly University Mori Lab
   
   @article{su2021roformer,
    title={Roformer: Enhanced transformer with rotary position embedding},
    author={Su, Jianlin and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng},
    journal={arXiv preprint arXiv:2104.09864},
    year={2021}
  }