Electra model class support loading weights from other types of BERTs
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Electra model class support loading weights from other types of BERTs(roberta, bert, etc..).
Most of them share the same backbone, so is it possible to support this features? 

Thanks.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

Hello, thanks for opening an issue and feature request! Could you enlighten me as to why you'd like that feature? 

I haven't thought deeply about it, but you're right that these models do share a very similar architecture! As we try to have our tools be as model agnostic as possible, I'm wondering what purpose loading BERT's weight in ELECTRA would serve, vs. simply loading BERT's weights in BERT?<|||||>Hi, my purpose is that I need a "post-train" phase between "pre-train" and "fine-tune" for downstream domain adaptation. Due to limited pre-trained resource, I have to adapt a pre-train model with faster electra training method to transfer an excellent pre-train models to downstream domain, so that I could fine-tuning with different tasks, or projects.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.