Continued training of the original BERT models (not to PyTorch)
# 🚀 Feature request

I am looking for continued training of original tiny-bert with my own raw data using masked language modelling. 
But I want the final model in TF.

## Motivation

I tried this [LM](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py), but this only works from pytorch to pytorch.

I tried with using original ckpt and from_tf, it always results in a .h5 error 

Please let me know I can explain more OR help in any way.

Basically, we should be able to use tf weights >> masked language modelling >> and have a domain specific pre-trained Tensorflow language model.



You should first convert your checkpoint to a huggingface checkpoint. You can check the docs on how to do that [here](https://huggingface.co/transformers/converting_tensorflow_models.html#bert).<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.