First of all, sorry if I am missing something because this is my first post on the forum.
I have thought that users / teams that train and share BERT like models could share the formatted corpus in order to other users to be able to use it in new models such as Electra easily saving time and computation (as the corpus format is the same).
Maybe HF could store and share formatted corpus or just the users/team can provide it.
The idea comes because I have seen several BERT like models in the hub for different languages and I would like to bring it to Electra as it needs less resources for pretraining and I (and maybe many users) can deal with it.