pytorch t5 seq2seq transformers text2text-generation
.gitattributes README.md config.json pytorch_model.bin special_tokens_map.json spiece.model tokenizer_config.json
# T5-base pretrained on CORD-19 dataset

The model has been pretrained on text and abstracts from the CORD-19 dataset, using a manually implemented denoising objetive similar to the original T5 denoising objective.

Model needs to be finetuned on downstream tasks.

Code avaliable in github: [https://github.com/manueldeprada/Pretraining-T5-PyTorch-Lightning](https://github.com/manueldeprada/Pretraining-T5-PyTorch-Lightning).