DialoGPT- cannot increase number of conversation turns
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: Linux-5.4.0-58-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.6.13
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten @LysandreJik

## Information

Model I am using (Bert, XLNet ...): DialoGPT

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

I fine-tuned DialoGPT on some sitcom subtitles, and now I am trying to chat with it using the commands listed [here](https://gist.github.com/albusdemens/9cd5602f088720e403f84038e088d696) (in the example, I use the Microsoft fine-tuned model).  

1. When I increase the number of conversation turns from 5 to 10, everything goes OK with the Microsoft model. 
2. If I increase the number of conversation turns to 10 using my fine-tuned model, reply number six is something like 

```
>> User:hello
OurBot: Hello.
>> User:how are things?
OurBot: They're fine.
>> User:cool. Did you have lunch already?
OurBot: I did, actually.
>> User:what did you have?
OurBot: Oh, um, i just wanted to toast.
>> User:where?
**OurBot: !!!hello!**
```

## Expected behavior

Using the DialoGPT model I fine-tuned, I would like to be able to have the same number of turns that I have using the Microsoft model. 

Hey @albusdemens,

Sorry I don't follow here completely. Does the model crash after 6 turns or does it just give a qualitatively bad answer?<|||||>Hey @patrickvonplaten, the second option (the quality of the answers is noticeably worse). Often I get outputs like `?!!`, `!.!!?!?` and similar. Usually low quality outputs don't show up in the first 3-4 conversation turns. When I use the Microsoft model instead, I don't get low-quality results after a few conversation turns. 

 Is there a way I can fix this?<|||||>This sounds very much like the model wasn't trained on long conversations to me...I'm not sure whether it's possible to enforce better quality without retraining the model<|||||>Thanks for your reply! Besides improving the quality of the training data,
do you think I should also increase the number of epochs?

On Tue, Mar 30, 2021, 6:15 AM Patrick von Platen ***@***.***>
wrote:

> This sounds very much like the model wasn't trained on long conversations
> to me...I'm not sure whether it's possible to enforce better quality
> without retraining the model
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/10742#issuecomment-809942835>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACDP25QF63SYDYBES5FT5TTGFUB5ANCNFSM4ZITFYLA>
> .
>
