Request: Providing gen_kwargs for Seq2SeqTrainer
### Feature request

As far as I am aware, there is currently no way to customize the generation behavior when generation is run as part of the training loop. `Seq2SeqTrainer.predict()` takes a `gen_kwargs` argument to pass along to `generate()`, but there is no way to pass `gen_kwargs` to `predict()` when predict is called automatically as part of the training loop.

### Motivation

Customizing generation may be necessary, doing this in the training loop as opposed to after training would be ideal.

### Your contribution

I'm happy to contribute, the main question is how this could be elegantly integrated into the current Trainer API.
May be of interest to @gante @sgugger <|||||>You can customize any generation kwarg by passing them to the model configuration (soon there will be a generation configuration that will make this clearer).<|||||>Ah I wasn't aware of that. It doesn't have all the generation parameters though, in my case I wanted to use prefix_allowed_tokens_fn, which model config doesn't seem to have. It's easily solved with a trainer subclass for me though, and it's probably a specific enough use case that the API doesn't need to be modified. Since the most common params can be used that way, I'll consider this resolved.<|||||>@atyshka I have the same issue using `prefix_allowed_tokens_fn` for Seq2SeqTrainer. Can you share how you solved it through trainer subclass?