One can use [T5ForConditionalGeneration] (or the Tensorflow/Flax variant), which includes the
language modeling head on top of the decoder.
Unsupervised denoising training
In this setup, spans of the input sequence are masked by so-called sentinel tokens (a.k.a unique mask tokens) and
  the output sequence is formed as a concatenation of the same sentinel tokens and the real masked tokens. Each
  sentinel token represents a unique mask token for this sentence and should start with <extra_id_0>,
  <extra_id_1>, ... up to <extra_id_99>. As a default, 100 sentinel tokens are available in
  [T5Tokenizer].
For instance, the sentence "The cute dog walks in the park" with the masks put on "cute dog" and "the" should be
  processed as follows:
```python
  from transformers import T5Tokenizer, T5ForConditionalGeneration
