Convert BERT to RoBERTa
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

Given that RoBERTa outperformed BERT on several tasks, yet having a slight architecture modification, I want to know if it is possible to convert a BERT pretrained model to RoBERTa. 

I am working with a BERT pretrained model on a domain specific corpora, given that I don't have the resources to train RoBERTa from scratch I want to know if I can convert the model to RoBERTa. If yes, how do I go about it?
The differences between the BERT and RoBERTa model are the following:

- Different pre-training (larger batch size for RoBERTa, no NSP, no token type ids ...)
- Different tokenizer

The model architecture is exactly the same. The only real difference after the pre-training is the difference in tokenization. Since you're working with a BERT model that was pre-trained, you  unfortunately won't be able to change the tokenizer now from a WordPiece (BERT) to a Byte-level BPE (RoBERTa).