Yes, great idea!
You could try to download the google cloud ckpt files then
python src/transformers/convert_t5_original_tf_checkpoint_to_pytorch.py --tf_checkpoint_path FIXME/model.ckpt-1014600  --pytorch_dump_path FIXME  --config_file t5-base-config.json
transformers-cli upload FIXME

where FIXME is the new model name.
Post a github issue if that breaks!