I guess you can use model_kwargs argument for .generate() function.
put encoder_outputs to model_kwargs, where encoder_outputs is encoder output of encoder-decoder model (like T5 and BART). I looked at the code and think this is the way.