Hi @nickmuchi, thanks for the bug report!
Indeed, you’re right that this model only has weights for PyTorch. However, you can load it in TensorFlow using the from_pt argument as follows:
from transformers import TFAutoModelForSeq2SeqLM

model = TFAutoModelForSeq2SeqLM.from_pretrained(model_checkpoint, from_pt=True)

In the meantime, I’ll post a fix on the website - thanks!