readme: ---
  language: 
  - zh
  
  pipeline_tag: "fill-mask"
  
  widget:
  - text: "ba黎系[MASK]国的首都"
    example_title: "Adversarial Attack Test"
  ---
  
  # RoCBert
  
  ## Introduction
  
  RoCBert is a pretrained Chinese language model that is robust under various forms of adversarial attacks proposed by WeChatAI in 2022, 
  
  More detail: https://aclanthology.org/2022.acl-long.65.pdf
  
  Pretrained code: https://github.com/sww9370/RoCBert
  
  ## How to use
  ```Python
  # pip install transformers>=4.25.1
  
  from transformers import AutoTokenizer, AutoModel
  tokenizer = AutoTokenizer.from_pretrained("weiweishi/roc-bert-base-zh")
  model = AutoModel.from_pretrained("weiweishi/roc-bert-base-zh")
  ```
  
  ## Citation
  
  ```bibtex
  @inproceedings{su2022rocbert,
    title={RoCBert: Robust Chinese Bert with Multimodal Contrastive Pretraining},
    author={Su, Hui and Shi, Weiwei and Shen, Xiaoyu and Xiao, Zhou and Ji, Tuo and Fang, Jiarui and Zhou, Jie},
    booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
    pages={921--931},
    year={2022}
  }
  ```