using BERT as pretraining with custom classifier
Hi
I need to pretrain the bert on one dataset and finetune it then on other datasets, so basically
removing classifier from first part and substitute it with a new one with the specific number of 
labels, currently with current codes, it will be error to do it when loading pretrained model,
could you please assist me how I can do this?
thanks
Best
Julia 

    model = model_class.from_pretrained(args.model_name_or_path, from_tf=bool('.ckpt' in args.model_name_or_path), config=config)
  File "julia/libs/anaconda3/envs/transformers/lib/python3.5/site-packages/pytorch_transformers/modeling_utils.py", line 461, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for BertRUBIForSequenceClassification:
	size mismatch for classifier.weight: copying a param with shape torch.Size([174, 768]) from checkpoint, the shape in current model is torch.Size([3, 768]).
	size mismatch for classifier.bias: copying a param with shape torch.Size([174]) from checkpoint, the shape in current model is torch.Size([3]).

Hi, it seems you have saved a model with a classification head of dimension `174 x 768`. You're then trying to load this model with a different classification head of dimension `3 x 768`, is that correct?

If you are trying to save/load the model without the classification head, you can simply save the BertModel **without the classification head**, and then load it from here.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@LysandreJik : I have the same problem, I trained the ner model and now want to fine tune on other datasets.
`Error(s) in loading state_dict for XLMRobertaForTokenClassification:
	size mismatch for classifier.weight: copying a param with shape torch.Size([9, 768]) from checkpoint, the shape in current model is torch.Size([24, 768]).
	size mismatch for classifier.bias: copying a param with shape torch.Size([9]) from checkpoint, the shape in current model is torch.Size([24])`

Can you please let me know how to "If you are trying to save/load the model without the classification head, you can simply save the BertModel without the classification head, and then load it from here." 