ELECTRA for Portuguese
Currently, there is no ELECTRA or ELECTRA Large model that was trained from scratch for Portuguese on the hub: Hugging Face – The AI community building the future. 3. For this project, the goal is to create a ELECTRA model for just the Portuguese language.

Model
A randomly initialized ELECTRA model

Available training scripts
A masked language modeling script for Flax is available here 4. It can be used pretty much without any required code changes.

(Optional) Desired project outcome
The desired project output is a strong ELECTRA model in Portuguese.