GPT2: discrepancy between `inputs_embeds` and `input_ids` when the input sentence has length = 1 
Calling GPT2 with `inputs_embeds` vs calling it with `input_ids` leads to different outcomes when the input sentence has length 1. See the script to reproduce the issue. 

## Environment info
- `transformers` version: tried 4.2.1 and 4.8.2 
- Platform: MacOS and Ubuntu 
- Python version: 3.7 
- PyTorch version (GPU?): 1.9.0
- Using GPU in script?: tried GPU and CPU 
- Using distributed or parallel set-up in script?: Nope 

### Who can help
@patrickvonplaten, @LysandreJik

## Information
Model I am using (Bert, XLNet ...): GPT2 


## To reproduce

```python 
model_name = 'gpt2'
print(" ==> Loading the models . . . ")
model = GPT2LMHeadModel.from_pretrained(model_name, output_hidden_states=True)
model.to(device)
model.eval()
tokenizer = GPT2Tokenizer.from_pretrained(model_name)

batch_size = 5
prefix_length = 1
for sentence_len in range(10, 0, -1):
    sentence_str = " ".join(["good"] * sentence_len)
    input_ids = tokenizer([sentence_str] * batch_size, return_tensors='pt')['input_ids'].to(device)
    inputs_embeds = model.transformer.wte(input_ids).squeeze()

    out1 = model(inputs_embeds=inputs_embeds)
    out2 = model(input_ids=input_ids)

    diff = torch.abs(torch.mean(out1.logits - out2.logits))
    print(f" - sentence length: {sentence_len} <-> {diff}")
```

which gives me the following: 
```
 - sentence length: 10 <-> 0.0
 - sentence length: 9 <-> 0.0
 - sentence length: 8 <-> 0.0
 - sentence length: 7 <-> 0.0
 - sentence length: 6 <-> 0.0
 - sentence length: 5 <-> 0.0
 - sentence length: 4 <-> 0.0
 - sentence length: 3 <-> 0.0
 - sentence length: 2 <-> 0.0
 - sentence length: 1 <-> 30.124570846557617
```

The last line is the surprise: when the input sentence has length == 1, `inputs_embeds` and `input_ids` lead to different outcomes. 

FYI @qkaren
Hello! I think the issue here lies in this line: `inputs_embeds = model.transformer.wte(input_ids).squeeze()`. You're calling `squeeze` on the inputs_embeds, which are of dimension `(5, 1, 768)`, which correspond to `(batch_size, sequence_length, hidden_size)`. Unfortunately, you're squeezing out the sequence length dimension (which is always required), which the model needs in order to construct the attention mask.

Remove the `.squeeze` or specify which dimension it should squeeze and you should be fine.