Some Models do not support gradient checkpointing
Thanks for this wonderful library. 

I found some models do not support gradient_checkpointing, which I believe is a very important feature. For example,

ElectraModel: ElectraConfig has no gradient_checkpointing option but ElectraModel will use gradient_checkpointing if config.gradient_checkpointing = True
DistillBERT: DistillBertConfig has no gradient_checkpointing option and DistillBertModel does not support gradient_checkpointing.

I assume all transformer-based models should be able to support gradient_checkpointing.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.