T5 supervised denoising task
# 🚀 Feature request

Hi everyone!

I'm experimenting with T5 and I would like to fine-tune a specific pre-trained model of mine for tackling the 'filling the mask' task. To be clear, I have the following: I love \<mask> and Mario. Where \<mask> can be a single token or span.
At the moment I framed the problem in this way:
   - input: I love <extra_id_0> and Mario.
   - output/label: luca

The task I want to tackle is different from the canonical unsupervised one where I was able to perform it correctly. 
Do you think that the discussed framing presented above is enough?

From the result that I got,  it doesn't seem so.

@patrickvonplaten Any news?<|||||>Yeah this looks good to me: 

input_ids: I love <extra_id_0> and Mario.
decoder_input_ids: `decoder_start_token_id`
output/label: luca <EOS><|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
