Hi all.
I’m creating a pretrianed T5 model with:
T5ForConditionalGeneration.from_pretrained("t5-small")

How is this model pretrained? It seems to me that the model weights I get here were trained at least on the GLUE dataset (and probably others).
I’d like it to only be pretrained on C4. Are those weights around somewhere? How do I get a model pretrained that way?
Thanks!