text classification reuse without classifier
thanks in advance.
I want to train first more data, it mans more label.
(its like pretraining)

after that I choose some label only, and train again.

But error happened during below code

tokenizer = RobertaTokenizer.from_pretrained(pretrained_path, do_lower_case=False)
model = RobertaForSequenceClassification.from_pretrained(pretrained_path, num_labels=10)

error message is like below.
Error(s) in loading state_dict for RobertaForSequenceClassification:
	size mismatch for classifier.out_proj.weight: copying a param with shape torch.Size([19, 768]) from checkpoint, the shape in current model is torch.Size([10, 768]).
	size mismatch for classifier.out_proj.bias: copying a param with shape torch.Size([19]) from checkpoint, the shape in current model is torch.Size([10]).

how can I do it??
Hi! This doesn't currently work, because it's trying to instantiate a classification layer with 10 labels, while the checkpoint's classification layer has 19 labels. 

What you want is to keep the base layers, but ignore the classification layers. In order to do so, you can load your checkpoint in a base `RobertaModel`. This will only load the base model. You can then save that model to a checkpoint, and load that checkpoint from a `RobertaForSequenceClassification` model.

Here's how to do this:

```py
from transformers import RobertaForSequenceClassification, RobertaConfig, RobertaModel

model = RobertaModel.from_pretrained(pretrained_path)
model.save_pretrained(f"{pretrained_path}-base-model")

config = RobertaConfig(num_labels=10)
model = RobertaForSequenceClassification.from_pretrained(f"{pretrained_path}-base-model", config=config)
```

You should see the output:

```
Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at here-base-model and are newly initialized: ['classifier.dense.weight', 'classifier.out_proj.bias', 'classifier.out_proj.weight', 'classifier.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

This means that it has correctly loaded the entire base model, and has randomly initialized the classifier weights. Hope that helps!