Why T5 do not generate the whole next sentence as one of the pretrain loss?
T5 has the next-sentence-predict loss. 
And the next-sentence-predict loss is not generating the whole next sentence but generating is_next or not_next token.

Thank you very much.
Hi @guotong1988  what do you exactly mean by not generating the whole next sentence ?<|||||>Sorry, my mistake.
Input the previous sentence and generate the next sentence, instead of input the two sentences.<|||||>I don't really understand the question here. Are you asking why T5 was not trained with the objective to generate the complete next sentence? <|||||>Thank you for your reply.
Yes. I am asking why T5 was not pre-trained with the objective to generate the complete next sentence?

Do you think generating the whole next sentence as one of the pretrain loss will improve pretrain performance?<|||||>![image](https://user-images.githubusercontent.com/4702353/86194695-c5af5980-bb81-11ea-98d4-487355f8bb60.png)
https://github.com/google-research/text-to-text-transfer-transformer/issues/286