readme: ---
  license: apache-2.0
  
  widget:
  - text: 横浜国立大学は日本の[MASK]県にある。
  ---
  
  
  This is RoBERTa model pretrained on texts in the Japanese language.
  
  3.45GB wikipedia text
  
   trained 1.65M step
   
    use the sentencepiece tokenizer.
    
    If you want to fine-tune model. Please use 
  ```python
  
  
  from transformers import BertTokenizer, RobertaModel
  BertTokenizer.from_pretrained('')
  RoBERTModel.from_pretrained('')
  ```
    
    
    The accuracy in JGLUE-marc_ja-v1.0 binary sentiment classification 95.4%
    
    
   Contribute by Yokohama Nationaly University Mori Lab
   
   @article{liu2019roberta,  
    title={Roberta: A robustly optimized bert pretraining approach},  
    author={Liu, Yinhan and Ott, Myle and Goyal, Naman and Du, Jingfei and Joshi, Mandar and Chen, Danqi and Levy, Omer and Lewis, Mike and Zettlemoyer, Luke and Stoyanov, 
    Veselin},  
    journal={arXiv preprint arXiv:1907.11692},  
    year={2019}  
  }  