readme: ---
  language: en
  ---
  
  # ERNIE-2.0
  
  ## Introduction
  
  ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
  which builds and learns incrementally pre-training tasks through constant multi-task learning. 
  Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 
  
  More detail: https://arxiv.org/abs/1907.12412
  
  ## Released Model Info
  
  This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
  a series of experiments have been conducted to check the accuracy of the conversion.
  
  - Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE
  - Pytorch Conversion repo:  https://github.com/nghuyong/ERNIE-Pytorch
  
  ## How to use
  ```Python
  from transformers import AutoTokenizer, AutoModel
  tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-2.0-base-en")
  model = AutoModel.from_pretrained("nghuyong/ernie-2.0-base-en")
  ```
  
  ## Citation
  
  ```bibtex
  @article{sun2019ernie20,
    title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding},
    author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng},
    journal={arXiv preprint arXiv:1907.12412},
    year={2019} 
  }
  ```
  