 Chinese small pre-trained model MiniRBT  In order to further promote the research and development of Chinese information processing, we launched a Chinese small pre-training model MiniRBT based on the self-developed knowledge distillation tool TextBrewer, combined with Whole Word Masking technology and Knowledge Distillation technology.  This repository is developed based on：https://github.com/iflytek/MiniRBT  You may also interested in, - Chinese LERT: https://github.com/ymcui/LERT - Chinese PERT: https://github.com/ymcui/PERT - Chinese MacBERT: https://github.com/ymcui/MacBERT - Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA - Chinese XLNet: https://github.com/ymcui/Chinese-XLNet - Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer  More resources by HFL: https://github.com/iflytek/HFL-Anthology