Save underlying BertModel only
I currently have a custom model on top of a pretrained BERT Model, which is effectively just dropout and a linear layer on top for classification. I want to be able to train this classifier, updating the underlying weights, but then to save the `BERTModel` underneath (w/o linear classification layer) so that I can load from file and use this as my input to the same custom model. Is there a way I can access and save the underlying transformer to then be used again like this?

Code for clarification:
```
class BERTBinaryClassifier(torch.nn.Module):

    def __init__(self, model_name_or_path: str):
        super(BERTBinaryClassifier, self).__init__()
        self.bert = ModelSelection.get_model(
            model_name=model_name_or_path)
        self.drop = torch.nn.Dropout(p=0.3)
        self.out = torch.nn.Linear(self.bert.config.hidden_size, 2).cuda()

    def forward(self, inputs):
        logits, embs, *_ = self.bert(**inputs)
        output = self.drop(logits)
        return self.out(output), embs
```

Right now I'm just outputting the logits and hidden_state for future usage but I'd like to be able to use this same function to effectively load a `BertModel` like I do at the start here with `self.bert` (which just loads `BertModel.from_pretrained` and save my Pytorch classifier as a whole, separately.

Would it be as simple as say accessing `self.bert.bert` and then saving it that way?
If your bert model is an instance of `BertModel` then you should be able to save it using `self.your_model.bert.save_pretrained(path)` and load using `.from_pretrained`<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.