Hi Patrick, I am also interested in training Electra for Portuguese. I don’t know if this is currently under development or already has been done, if not, I would like to implement this pretraining script and train the model for Portuguese.