Thanks @JulesBelveze. I guess you did the right choice with the change from T5ForConditionalGeneration to MT5ForConditionalGeneration in onnx_exporter.py of fastT5 (but it means that we need to manually change the official fastT5 library…).
In the case of ByT5, it is different as the ByT5Tokenizer 1 is not part of transformers=="4.6.1".
How to convert a ByT5 model to the ONNX format?