readme: # SentiWSP
  ## For paper: Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis
  We propose **SentiWSP**, a novel **Senti**ment-aware pre-trained language model with combined **W**ord-level and **S**entence-level **P**re-training tasks.
  The word level pre-training task detects replaced sentiment words, via a generator-discriminator framework, to enhance the PLM's knowledge about sentiment words.
  The sentence level pre-training task further strengthens the discriminator via a contrastive learning framework, with similar sentences as negative samples, to encode sentiments in a sentence.
  
  ## Fine-tunning
  You can also load our model in huggingface ([https://huggingface.co/shuaifan/SentiWSP-base](https://huggingface.co/shuaifan/SentiWSP-base)) to fine-tunning in sentiment analysis tasks:
  ```python
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
  import torch
  
  tokenizer = AutoTokenizer.from_pretrained("shuaifan/SentiWSP-base")
  model = AutoModelForSequenceClassification.from_pretrained("shuaifan/SentiWSP-base")
  ```