Generate does not take into account config.decoder.eos_token_id
As reported by some people (see https://github.com/NielsRogge/Transformers-Tutorials/issues/53 and on the [forum](https://discuss.huggingface.co/t/trocr-repeated-generation/12361)), the `generate()` method currently does not take into account `config.decoder.eos_token_id`, only` config.eos_token_id` to properly stop generation.

Hence, models that are made using `EncoderDecoderModel`/`VisionEncoderDecoderModel`/`SpeechEncoderDecoderModel` will not properly stop generation if `config.eos_token_id` is not set.

cc @patrickvonplaten @patil-suraj 
Hmm, yeah I think I'm fine with adding some `if - statements` to the `generate()` method<|||||>Do you want to open a PR for it? :-)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>i will try to fix it<|||||>I pulled a request https://github.com/huggingface/transformers/pull/15403, but the ci failed. By analysing the ci failture log, I find that it has a hidden logic, if you don't pass a eos_pos_id, you want the model to generate until max-length. That is what the code do
https://github.com/huggingface/transformers/blob/16d4acbfdb547cb922361ba07a13de12e1503fb8/tests/test_modeling_encoder_decoder.py#L404
So, adding self.config.decoder.eos_pos_id simply is not enough. 
cc @NielsRogge @patrickvonplaten <|||||>Hi, the above pull request mentioned offhandedly that this had been fixed, is that the case or is this still open as indicated?<|||||>Seems like this got fixed, closing the issue.