[GPT2] Propose fix for #21080
# What does this PR do?
Propose fix for #21080.
Modify the default generation of the positional ids for the `gpt-2` as well as the `decidion_transformer` model. 
In the issue @LysandreJik proposed to update the doc, but when I checked it seemed like this would only affect 2 models, and is backward compatible: 
- potential impact is only for people that where using batched padded input when generating with a single sentence. This means that their output will now be corrected
- default behavior is kept if not attention mask is given. 

If this is not acceptable, I am also glad to add a warning when creating the positional ids.
_The documentation is not available anymore as the PR was closed or merged._<|||||>I'd like to summon the generation expert @gante to ask what he thinks of the PR<|||||>The summon has been heard 📣 

### TL;DR

I approve the change 👍 But we need to have a second look at the position embeddings, I suspect there are several bugs in the codebase (see why below)

### Context
This PR actually took me on a long trip, whose findings I summarize below:
1. I saw that @ArthurZucker wrote `this would only affect 2 models`, and my first thought was `what about GPT-J`?
2. Then I saw #21869 (remove `position_ids` input from GPT-J). From the TF XLA `.generate` transition, I remember that getting the `position_ids` to work with GPT-J was a nice piece of work, and I remember that it made a difference in the outputs. See the example below

<details>
  <summary>GPT-J + positions_ids</summary>
  
  ```py
  from transformers import TFAutoModelForCausalLM, AutoTokenizer
  import tensorflow as tf
  
  tf.keras.backend.set_floatx('float16')
  
  tok = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B", padding_side="left")
  model = TFAutoModelForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", from_pt=True)
  tok.pad_token = tok.eos_token
  model.generation_config.pad_token_id = model.generation_config.eos_token_id
  
  inputs = tok(["and the prime minister"], return_tensors="tf", padding=True)
  out_1 = model(**inputs)
  out_2 = model(**inputs)
  
  position_ids = tf.math.cumsum(inputs.attention_mask, axis=-1, exclusive=True)
  out_3 = model(**inputs, position_ids=position_ids + 10)
  
  print(tf.reduce_max(tf.abs(out_1.logits[:, -1, :] - out_2.logits[:, -1, :]))) # tf.Tensor(0.0, shape=(), dtype=float16)
  print(tf.reduce_max(tf.abs(out_1.logits[:, -1, :] - out_3.logits[:, -1, :]))) # tf.Tensor(0.01563, shape=(), dtype=float16)
  ```
</details>


3. Despite the above, and looking at this PR, I think what @ArthurZucker wrote here is the way to go. In `prepare_inputs_for_generation`, we were computing `position_ids` from the `attention_mask` anyways. If we pass the logic from there to the forward pass, we cut a source of bugs (users trying to generate without `.generate()`) 👍 
4. GPT-J needs to be fixed (see TF/FLAX) :p To be clear, the issue is long-standing and not a result of #21869 !
5. We should double-check at least the main models. Some models, like `OPT`, are okay, as they compute the position embedding directly from the attention mask. Others, like `Codegen`, suffer from the same problem as `GPT-J`.
 
<|||||>@gante I was coincidentally just having this problem with codegen and so have opened #22069 following your hints above.<|||||>Thanks for the in depth review @gante ! <|||||>Will re-open this with a fix for the cross PT-TF tests. The TF code has to be modified as otherwise the default positional ids are wrong. <|||||>Sorry for missing that the tf version also needed an update on this one! <|||||>Thank you @ArthurZucker . No worry - we didn't detect this because the PT/TF cross tests in the corresponding TF model test files are not fetched by the test fetcher script. The current version will only detect the tests in the (modified + involved indirectly) PyTorch modeling/test files.