  to the documentation of byT5 which can be found here.
All checkpoints can be found on the hub.
This model was contributed by thomwolf. The original code can be found here.
Training
T5 is an encoder-decoder model and converts all NLP problems into a text-to-text format. It is trained using teacher
forcing. This means that for training, we always need an input sequence and a corresponding target sequence. The input
sequence is fed to the model using input_ids. The target sequence is shifted to the right, i.e., prepended by a
start-sequence token and fed to the decoder using the decoder_input_ids. In teacher-forcing style, the target
sequence is then appended by the EOS token and corresponds to the labels. The PAD token is hereby used as the
start-sequence token. T5 can be trained / fine-tuned both in a supervised and unsupervised fashion.
