Fix input ids can be none attn mask
Make sure `batch_size` is correct for gpt2 and ctrl - these models need a slightly different behavior since the shape of `input_ids` can change depending on whether the past variable is inserted or not. 
See also:
PR: https://github.com/huggingface/transformers/pull/3033
and its issue:
https://github.com/huggingface/transformers/issues/3031
Thanks for pointing out @lazarevskiVsg and @julien-c !<|||||>@julien-c , will merge right away - small change!<|||||>Why don’t you just use input_shape (which is always defined), to be consistent with other models?<|||||>> Why don’t you just use input_shape (which is always defined), to be consistent with other models?

The problem is that GPT2 and CTRL have a different behavior (and the `input_ids` shape changes) when the `past` variable is inserted which previously led to problem when the attention_mask is inserted as well: 
#3031 

Therefore this slightly weird implementation.<|||||>But in your code in this PR, batch_size is always input_shape[0] anyways, no?<|||||>> But in your code in this PR, batch_size is always `input_shape[0]` anyways, no?

I think in the case of CTRL and GPT2, it's actually a bigger inconsistency:

Let's say we have an input_ids tensor of shape `[batch_size, sequence_length] = [5, 4]`.

We call `GPT2Model` and save the last `output embeddings = outputs[0][:, -1, :]` **and** the `past` key/value states to speed up decoding = `outputs[1]`

Now if we want to use `past` GPT expects the `input_ids` to be of shape `[batch_size, 1] .squeezed(-1) = [batch_size]`. Therefore we have to adapt the attention mask here differently than in other models. Which is weird (and a bit suboptimal in my opinion in GPT's and CTLR's API) is that the shape of `input_ids` differs depending on whether `past` is None or not. 

@julien-c 