unconditional generation with run_generation.py
## ❓ Questions & Help

Is it possible to have unconditional generation with ``` run_generation.py``` ? I realized the previous script ```run_gpt2.py``` had this option.
Can we use the same ```start_token``` trick?
Hi, by unconditional generation do you mean generating a sequence from no context? If so, if using GPT-2, you can set your initial context to be: `<|endoftext|>`. This will generate sequences with no other initial context.

You could do so like this:

```
from pytorch_transformers import GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
context = [tokenizer.encoder["<|endoftext|>"]]
```<|||||>@LysandreJik Yeah I did that but It is not performing good, it generated the start like this:
```
"<|endoftext|>ingly anticipation passations, out he didn't realize that any or products in order to stand on. As Eric's disappointment, he threw into those o-bag's fanware vugeless chainsas. Finally, Chris went on a hob, and grabbed the crurne cartocos juice!"
```
I wonder if I can add a starting special token (sth like [CLS]) to my inputs and finetune gpt2 with this added vocabulary?
I have asked this [here](https://github.com/huggingface/pytorch-transformers/issues/1145) though<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> @LysandreJik Yeah I did that but It is not performing good, it generated the start like this:
> 
> ```
> "<|endoftext|>ingly anticipation passations, out he didn't realize that any or products in order to stand on. As Eric's disappointment, he threw into those o-bag's fanware vugeless chainsas. Finally, Chris went on a hob, and grabbed the crurne cartocos juice!"
> ```
> 
> I wonder if I can add a starting special token (sth like [CLS]) to my inputs and finetune gpt2 with this added vocabulary?
> I have asked this [here](https://github.com/huggingface/pytorch-transformers/issues/1145) though

Hi Have you figured out this issue?