gpt2-large and gpt2-xl behave strangely with pad tokens
### System Info

- `transformers` version: 4.18.0
- Platform: Linux-4.18.0-425.13.1.el8_7.x86_64-x86_64-with-centos-8.7-Green_Obsidian
- Python version: 3.6.8
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.10.2+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: False

### Who can help?

@ArthurZucker @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
pretrained = 'gpt2-large'
model = GPT2LMHeadModel.from_pretrained(pretrained).to('cuda')
tokenizer = AutoTokenizer.from_pretrained(pretrained, padding_side='left')
tokenizer.add_special_tokens({'pad_token': <|endoftext|>})

torch.cuda.manual_seed_all(2266)

input0 = '<|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|>Austin knew Quinn intimately and they slept together many times. Why did Austin do this? (0) Hated Quinn, (1) Found QUinn attractive, (2) Ask Quinn on a date\nANSWER:'
input_ids = tokenizer(input0, return_tensors='pt').input_ids.to(device)
with torch.no_grad():
    output0 = model.generate(input_ids, max_new_tokens=16, top_k=20, pad_token_id=50256, eos_token_id=50256, do_sample=True, temperature=0.01)

input1 = 'Austin knew Quinn intimately and they slept together many times. Why did Austin do this? (0) Hated Quinn, (1) Found QUinn attractive, (2) Ask Quinn on a date\nANSWER:'
input_ids = tokenizer(input1, return_tensors='pt').input_ids.to(device)
with torch.no_grad():
    output1 = model.generate(input_ids, max_new_tokens=8, top_k=20, pad_token_id=50256, eos_token_id=50256, do_sample=True, temperature=0.01)
```

### Expected behavior

```
output0 = '\nThe The The The The'
output1 = ' Austin was jealous of Quinn's relationship'
```

pad_token works just fine with other gpt models such as gpt2, and gpt2-small.
Hi @boblus 
In order for your script to work, you need to properly set the attention mask by masking out the padding tokens and call generate with the attention mask<|||||>Do you mean `tokenizer.add_special_tokens({'pad_token': pad_token})`?
I have done this. I forgot to put that in my initial comment. Just added it.<|||||>I think that attention masks are created only if you pass multiple sentences to the tokenizer, in your case I think you may need to create it manually unless I am wrong and there is a simpler solution.

The below script worked for me:
```python
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

pretrained = 'gpt2-large'
device = 'cuda'
model = GPT2LMHeadModel.from_pretrained(pretrained).to('cuda')
tokenizer = GPT2Tokenizer.from_pretrained(pretrained)

torch.cuda.manual_seed_all(2266)

input0 = '<|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|><|endoftext|>Austin knew Quinn intimately and they slept together many times. Why did Austin do this? (0) Hated Quinn, (1) Found QUinn attractive, (2) Ask Quinn on a date\nANSWER:'

tokenizer.pad_token = tokenizer.eos_token
tokenizer.pad_token_id = tokenizer.eos_token_id

inputs = tokenizer(input0, return_tensors='pt').to(device)
inputs["attention_mask"] = inputs["input_ids"].ne(tokenizer.pad_token_id).long().to(device)
with torch.no_grad():
    output0 = model.generate(**inputs, max_new_tokens=16, top_k=20, pad_token_id=50256, eos_token_id=50256, do_sample=True, temperature=0.01)
    print(tokenizer.decode(output0[0]))

input1 = 'Austin knew Quinn intimately and they slept together many times. Why did Austin do this? (0) Hated Quinn, (1) Found QUinn attractive, (2) Ask Quinn on a date\nANSWER:'
input_ids = tokenizer(input1, return_tensors='pt').input_ids.to(device)
with torch.no_grad():
    output1 = model.generate(input_ids, max_new_tokens=8, top_k=20, pad_token_id=50256, eos_token_id=50256, do_sample=True, temperature=0.01)
    print(tokenizer.decode(output1[0]))
```<|||||>Thanks @younesbelkada, it works now. I thought setting `pad_token_id` in `model.generate()` is equivalent to setting the attention mask. And my codes worked well with other models, that's why it confused me.<|||||>linked to #22155  and #21080. GPT2 is an old model and does not necessarly create everything by default like our recent models