[T5] make decoder input ids optional for t5 training
- [x] Make `decoder_input_ids` optional when supplying `lm_labels` for `T5ForConditionalGeneration`
- [x] Add test
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=h1) Report
> Merging [#3521](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/33ef7002e17fe42b276dc6d36c07a3c39b1f09ed&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `95.23%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3521/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3521   +/-   ##
=======================================
  Coverage   77.80%   77.81%           
=======================================
  Files         100      100           
  Lines       17051    17069   +18     
=======================================
+ Hits        13267    13282   +15     
- Misses       3784     3787    +3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3521/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.15% <ø> (-0.18%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3521/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.81% <ø> (-0.14%)` | :arrow_down: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/3521/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `81.79% <95.23%> (+0.50%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=footer). Last update [33ef700...168bed0](https://codecov.io/gh/huggingface/transformers/pull/3521?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>@patrickvonplaten  Hi Patrick! Could you tell me what is the difference between decoder_input_ids and lm_labels for T5ForConditionalGeneration? For context: I am using T5ForConditionalGeneration for paraphrase generation. I am checking this code:https://github.com/ramsrigouthamg/Paraphrase-any-question-with-T5-Text-To-Text-Transfer-Transformer-/blob/master/t5-pretrained-question-paraphraser.ipynb He uses lm_labels with decoder_attention_mask. Thanks in advance!<|||||>@mengyahuUSTC-PU .      When calling the [forward()](https://github.com/ramsrigouthamg/Paraphrase-any-question-with-T5-Text-To-Text-Transfer-Transformer-/blob/9b26db2336d6077cc9d95bc28f123d32298aaf94/train.py#L66) , decoder_input_ids is None as follows:
```
             outputs = self(
            input_ids=batch["source_ids"],
            attention_mask=batch["source_mask"],
            lm_labels=lm_labels,
            decoder_attention_mask=batch['target_mask']
        )
```

decode_input_ids is derived from lm_labels if decode_input_ids is None. [decode_input_ids=](https://github.com/huggingface/transformers/blob/1aec991643a6fec0e7d504626fc68347fe93b658/src/transformers/modeling_t5.py#L1156)

I was wondering in what case I need to feed decode_input_ids.

