Encoder Decoder Model Generation Issue
Thank you for your great works!

I'm customizing Enc-Dec model, but it can't get additional parameters. 

How about modifying from below, 
https://github.com/huggingface/transformers/blob/0ddadbf0a82c3d9f3df864e1302e3f04e25c2967/src/transformers/models/encoder_decoder/modeling_encoder_decoder.py#L471

to
```
decoder_inputs = self.decoder.prepare_inputs_for_generation(input_ids, past=past, **kwargs)
```
Hi! It should be possible to do that, but could please explain the issue in more detail so we could better understand it? Thanks!<|||||>@patil-suraj @patrickvonplaten 
I did pull request https://github.com/huggingface/transformers/pull/13889, it's described below.
When implementing [Pointer Network], it needs to get tokens of the source text. But that returns only default keys(e. g. decoder_input_ids). And it's bound "prepare_inputs_for_generation" by "generation_utils", so it can't add other parameters.
So i add parameter "use_default_keys". Then variable "input_ids" can be extended from each language model head's  "prepare_inputs_for_generation" modefied by users.

Let's say, if using Bert2Bert model implementation of below, it can be getting "decoder_src_input_ids" on decoding when use **kwargs in parent function of "prepare_inputs_for_generation".
``` 
def prepare_inputs_for_generation(self, input_ids, past=None, attention_mask=None, **kwargs):
        input_shape = input_ids.shape
        # if model is used as a decoder in encoder-decoder model, the decoder attention mask is created on the fly
        if attention_mask is None:
            attention_mask = input_ids.new_ones(input_shape)

        # cut decoder_input_ids if past is used
        if past is not None:
            input_ids = input_ids[:, -1:]
        src_input_ids = kwargs.get('decoder_src_input_ids', None)
        return {"input_ids": input_ids, "attention_mask": attention_mask, "past_key_values": past, "decoder_src_input_ids": src_input_ids}
```


[Pointer Network]: https://arxiv.org/abs/1704.04368

Thank you for reply!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Closing due to inactivity on the pull request