  )
  labels = target_encoding.input_ids
# replace padding token id's of the labels by -100
  labels = torch.tensor(labels)
  labels[labels == tokenizer.pad_token_id] = -100
# forward pass
  loss = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels).loss
  ```
Additional training tips:
T5 models need a slightly higher learning rate than the default one set in the Trainer when using the AdamW
  optimizer. Typically, 1e-4 and 3e-4 work well for most problems (classification, summarization, translation, question
  answering, question generation). Note that T5 was pre-trained using the AdaFactor optimizer.
According to this forum post, task prefixes matter when
  (1) doing multi-task training (2) your task is similar or related to one of the supervised tasks used in T5's
  pre-training mixture (see Appendix D of the paper for the task prefixes
