  used).
If training on TPU, it is recommended to pad all examples of the dataset to the same length or make use of
  pad_to_multiple_of to have a small number of predefined bucket sizes to fit all examples in. Dynamically padding
  batches to the longest example is not recommended on TPU as it triggers a recompilation for every batch shape that is
  encountered during training thus significantly slowing down the training. only padding up to the longest example in a
  batch) leads to very slow training on TPU.
Inference
At inference time, it is recommended to use [~generation_utils.GenerationMixin.generate]. This
method takes care of encoding the input and feeding the encoded hidden states via cross-attention layers to the decoder
and auto-regressively generates the decoder output. Check out this blog post to know all the details about generating text with Transformers.
