Finetuning T5 Model
Hi there. I am trying to fine tune T5, but I have noticed your documentation gives conflicting instructions.

In modeling_t5.py you say
> To match pre-training, T5 input sequence should be formatted with [CLS] and [SEP] tokens

but then in tokenization_t5.py both of those tokens are set to None and the only tokens that are defined are EOS, UNK, and PAD.
Additionally, the actual T5 implementation makes no mention of SEP and CLS as far as I can tell.
Given that, could you clarify how we should be formatting our training data for HuggingFace's T5 implementation? Thank you!
Hi @jkangsta,

Thanks for posting your question here. The docstring was out of date and an in-detail description for T5 will be added here #3507 .