hey, @valhalla
I’ve created fastT5 87 library. it exports T5 model to onnx with
past_key_values.  the exported model supports generate() method.
for onnx models I was getting the inference speed boost of up to 5x for greedy and 3-4x for beam search. for more details check out the fastT5 repo 87.
currently, transformers library does not support exporting of t5 to onnx with past_key_values, you can fix this issue by following the guide in this notebook 27. created PR for this support here 9