[T5] Ignore sentinel indices for unsupervised denoising / masking objective?
The [docs](https://huggingface.co/transformers/model_doc/t5.html#training) state that the masked language modeling objective is simply

```
input_ids = tokenizer.encode('The <extra_id_0> walks in <extra_id_1> park', return_tensors='pt')
labels = tokenizer.encode('<extra_id_0> cute dog <extra_id_1> the <extra_id_2> </s>', return_tensors='pt')
model(input_ids=input_ids, labels=labels)
```

I was wondering if I need to manually set the `additional_special_tokens_ids` (corresponding to the `<extra_id_#>` sentinels) in the `labels` to `-100` during training so that they are ignored by the loss, as I believe would be the case for the `[MASK]` tokens in BERT? It seems that at least the `pad_token_id` is ignored in [`examples/seq2seq`](https://github.com/huggingface/transformers/blob/a09fe140c1c059baf05c4f97e5b4e83c719608db/examples/seq2seq/finetune.py#L153), but it's not clear if this ought to be true for the sentinels as well. My suspicion is _no_, but since there's no canonical MLM code for T5, I figured it was worth checking.

(I asked this in the forums and in a somewhat related issue, but was recommended to post here & tag @patrickvonplaten / @thomwolf)
Hey @ahoho - good question! 

I'm pretty confident that you should mask all sentinel tokens (with -100) and only compute the loss the "real" labels being "cute dog", "the"  and "</s>". 

Also they are definitely not automatically ignored as is done for the pad_token_id in `examples/seq2seq`

I could not find a more detailed explanation in the paper - so maybe @craffel could take a quick look as well and confirm (hope it's fine to tag you here Colin)<|||||>No need to treat the sentinel tokens specially (masking out their loss or otherwise). The model is trained to output both the sentinel tokens and the filled-in blanks.