readme: Distilled version of the [RoBERTa](https://huggingface.co/textattack/roberta-base-SST-2) model fine-tuned on the SST-2 part of the GLUE dataset. It was obtained from the "teacher" RoBERTa model by using task-specific knowledge distillation. Since the teacher was fine-tuned on the SST-2, the final model as well is ready to be used in sentiment analysis tasks.
  
  ## Modifications to the original RoBERTa model:
  
  The final distilled model was able to achieve 92% accuracy on the SST-2 dataset. Given the original RoBERTa achieves 94.8% accuracy on the same dataset with much more parameters (125M) and that the distilled model is nearly twice as fast as it is, the accuracy is an impressive result.
  
  
  ## Training Results after Hyperparameter Tuning
  |       Epoch       |  Training Loss  |  Validation Loss  |   Accuracy   |
  | ----------------- |   ------------  |     ---------     |  ----------  |
  |1                  |    0.144000     |   0.379220        |   0.907110   |
  |2                  |    0.108500     |   0.466671        |   0.911697   |
  |3                  |    0.078600     |   0.359551        |   0.915138   |
  |4                  |    0.057400     |   0.358214        |   0.920872   |
  
  
  ## Usage
  
  To use the model from the 🤗/transformers library
  
  ```python
  # !pip install transformers
  
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
  
  tokenizer = AutoTokenizer.from_pretrained("azizbarank/distilroberta-base-sst2-distilled")
  
  model = AutoModelForSequenceClassification.from_pretrained("azizbarank/distilroberta-base-sst2-distilled")
  ```