Hello,
Is it possible to load a pretrained Bert model but with a different type_vocab_size value?
For exmaple.:


The pretrained model: stsb-bert-base


The config.json of the stsb-bert-base model:



Now, I want to set the type_vocab_size to 20, is there a possible way that I can still use the pretrained  stsb-bert-base model as a start point to fine-tune on my own dataset? Or I have to train the model from scratch?


As a reference, I tried to loading the pretrained model with a modified type_vocab_size=20, but got an error as below:
“RuntimeError: Error(s) in loading state_dict for BertModel: size mismatch for bert.embeddings.token_type_embeddings.weight: copying a param with shape torch.Size([2, 768]) from checkpoint, the shape in current model is torch.Size([20, 768]).”


Thank you! 
Br,
Na