pretrain longT5
Could you please provide us with the steps of pretraining the longT5 on both MLM and PSG objectives?
Denoising rates and other details and steps? 
https://arxiv.org/pdf/2112.07916.pdf

All the requested information is on the paper. I would recommend sending questions over to https://github.com/google-research/longt5 since the authors will be far more able to answer specific questions.<|||||>@reelmath  thanks alot!  I already know that but the steps are in c++ script and no clear documentation to generate the corpus and using it to calculate loss. Since in Pegasus these are two losses. what is not clear do they calculate just one loss or two losses in longt5 during pretraining?
beside I want to pretrain the model from huggingface.
https://github.com/google-research/longt5/issues/7#issue-1319140379<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.