readme: ---
  tags:
    - conversational
  ---
  
  DialoGPT-small finetuned in the Switchboard Dialogue Act (SwDa) Corpus. The [repository](https://github.com/KonstSkouras/Switchboard-Corpus/tree/develop/) with additionally pre-processed SwDa dialogues of concatenated utterances per speaker, 80/10/10 train/val/test split and metadata, is a fork of the [Nathan Duran's repository](https://github.com/NathanDuran/Switchboard-Corpus). For finetuning the `train_dialogpt.ipynb` notebook from Nathan Cooper's [Tutorial](https://nathancooper.io/i-am-a-nerd/chatbot/deep-learning/gpt2/2020/05/12/chatbot-part-1.html) was used to finetune the model with slight modifications in Google Collab.
  
  History dialogue context = 5. 
  Number of utterances:  80704 (train set), 9749 (test set), 9616 (val set).
  Checkpoint-84000 after fine-tuning for 2 epochs with batch size 2.
  
  
  Evaluation perplexity in SwDa from 635.6993 (pre-trainded model) to 18.1693 (fine-tuned model).