readme: ---
  language: en
  ---
  
  # SKEP-
  
  ## Introduction
  
  SKEP (SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis) is proposed by Baidu in 2020,
  
  SKEP propose Sentiment Knowledge Enhanced Pre-training for sentiment analysis. Sentiment masking and three sentiment pre-training objectives are designed to incorporate various types of knowledge for pre-training model.
  
  More detail: https://aclanthology.org/2020.acl-main.374.pdf
  
  ## ⚠️ attention
  Compared with the full version of the ernie_2.0_skep_large_en, we lost the task_embeddings part in order to adapt to the Bert framework.
  
  
  
  ## Released Model Info
  
  |Model Name|Language|Model Structure|
  |:---:|:---:|:---:|
  |skep-ernie2-bert-large| English |Layer:24, Hidden:1024, Heads:24|
  
  This released pytorch model is converted from the officially released PaddlePaddle SKEP model and 
  a series of experiments have been conducted to check the accuracy of the conversion.
  
  - Official PaddlePaddle SKEP repo: 
    1. https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/transformers/skep
    2. https://github.com/baidu/Senta
  - Pytorch Conversion repo: Not released yet
  
  
  ## How to use
  ```Python
  from transformers import AutoTokenizer, AutoModel
  tokenizer = AutoTokenizer.from_pretrained("Yaxin/ernie_2.0_skep_large_en")
  model = AutoModel.from_pretrained("Yaxin/ernie_2.0_skep_large_en")
  ```
  
  ## Citation
  
  ```bibtex
  @article{tian2020skep,
    title={SKEP: Sentiment knowledge enhanced pre-training for sentiment analysis},
    author={Tian, Hao and Gao, Can and Xiao, Xinyan and Liu, Hao and He, Bolei and Wu, Hua and Wang, Haifeng and Wu, Feng},
    journal={arXiv preprint arXiv:2005.05635},
    year={2020}
  }
  
  ```
  
  ```
  reference:
  https://github.com/nghuyong/ERNIE-Pytorch
  
  ```
  