I modified https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py script few days ago for training electra from scratch. But there were some problems(maybe bugs) i had to solve for this task.
I modified https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py script few days ago for training electra from scratch. But there were some problems(maybe bugs) i had to solve for this task. 

Currently I´m setting up a clean running version for training a electra language model from scratch with an additional document classification head based on the script.

_Originally posted by @miketrimmel in https://github.com/huggingface/transformers/issues/4425#issuecomment-630715171_
location is currently not available...please share the exact location<|||||>Here's the folder containing the language modeling scripts: https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.