Fine-tuning distilled GPT-2
## ❓ Questions & Help

To my understanding, examples/run_lm_finetuning.py can be used to fine-tune the model to new data. How do I fine-tune a distilled GPT-2? To be precise, I assume that I can use the entire code, but I just need to import the right module. I tried importing DistilGPT2Config, DistilGPT2LMHeadModel, DistilGPT2Tokenizer, but it doesn't work out.
_DistilGPT2Config_, _DistilGPT2LMHeadModel_ and _DistilGPT2Tokenizer_  **don't exist**. In order to fine-tuning the DistilGPT2 model for LM, you can use the following settings of tokenizer, config and model:
**Tokenizer**:
```
> from transformers import GPT2Tokenizer
> tokenizer = GPT2Tokenizer.from_pretrained('distilgpt2', )
```
N.B: as said in the source code [here](https://github.com/huggingface/transformers/blob/35401fe50fa3e460b2a4422630b017f106c79e03/transformers/tokenization_gpt2.py), this tokenizer requires a space to start the input string, therefore the `encoding` and `tokenize` methods should be called with the `add_prefix_space` flag set to `True`. Otherwise, this tokenizer's `encode`, `decode`, and `tokenize` methods will not conserve the spaces at the beginning of a string: `tokenizer.decode(tokenizer.encode(" Hello")) = "Hello"`

**Config**:
```
> from transformers import GPT2Config
> config = GPT2Config.from_pretrained('distilgpt2')
```

**Model**:
```
> from transformers import GPT2LMHeadModel
> model = GPT2LMHeadModel.from_pretrained('distilgpt2')
```
N.B: for completeness, in order to use DistilGPT2 model, you have to use the following code: `model = GPT2Model.from_pretrained('distilgpt2')`.

> ##  Questions & Help
> To my understanding, examples/run_lm_finetuning.py can be used to fine-tune the model to new data. How do I fine-tune a distilled GPT-2? To be precise, I assume that I can use the entire code, but I just need to import the right module. I tried importing DistilGPT2Config, DistilGPT2LMHeadModel, DistilGPT2Tokenizer, but it doesn't work out.<|||||>It works. Thank you.