readme: ---
  language: 
  - zh
  tags:
  - bert
  license: "apache-2.0"
  ---
  
  # Please use 'Bert' related functions to load this model!
  
  ## Chinese small pre-trained model MiniRBT
  In order to further promote the research and development of Chinese information processing, we launched a Chinese small pre-training model MiniRBT based on the self-developed knowledge distillation tool TextBrewer, combined with Whole Word Masking technology and Knowledge Distillation technology.
  
  This repository is developed based on：https://github.com/iflytek/MiniRBT
  
  You may also interested in,
  - Chinese LERT: https://github.com/ymcui/LERT
  - Chinese PERT: https://github.com/ymcui/PERT
  - Chinese MacBERT: https://github.com/ymcui/MacBERT
  - Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA
  - Chinese XLNet: https://github.com/ymcui/Chinese-XLNet
  - Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer
  
  More resources by HFL: https://github.com/iflytek/HFL-Anthology