Hi guys,
It seems mT5 employs the T5.1.1 architecture (not the original T5 arch), as you can see from the name T5-XL and T5-XXL instead of T5-3B and T5-11B.
In this case, HF still doesn’t have implementation on this T5.1.1 yet. Please see : https://github.com/huggingface/transformers/issues/6285 31
UPDATED (Nov 17, 2020) : will be released soon by amazing Patrick – https://github.com/huggingface/transformers/pull/8552 9