how to save a model with additional layers, so it can be loaded using .from_pretrained()?
## Environment info

- `transformers` version: 2.3.0
- Platform: Linux-4.15.0-135-generic-x86_64-with-glibc2.27
- Python version:  3.8.12
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: YES (T4 TESLA)
- Using distributed or parallel set-up in script?: no

### Who can help
@LysandreJik

## Information

Model I am using Bert: Text-classification

## To reproduce

Steps to reproduce the behavior:

I have a custom model

`class BERTClass(torch.nn.Module):
    def __init__(self, num_labels=4):
        super(BERTClass, self).__init__()
        self.l1 = BertModel.from_pretrained('bert-base-multilingual-uncased')
        self.l2 = torch.mean
        self.l3 = torch.nn.Linear(768, num_labels)

    def forward(self, ids, masks, token_type_ids):        
        last_hidden_state, _ = self.l1(ids, attention_mask=masks, token_type_ids=token_type_ids)
        avg_pooling = self.l2(last_hidden_state, dim=1)
        output = self.l3(avg_pooling)
        
        return output`

How do I go about saving my custom model so that I can run BertModel.from_pretrained('./mycustomodel')

Do you have an example I can follow by chance?

Currently I am saving the model this way

`
model_2_save = model.module if hasattr(model, "module") else model

checkpoint = {
    'epoch': args.epochs,
    'num_labels': args.num_labels,
    'max_text_length': MAX_TEXT_LENGTH,
    'state_dict': model_2_save.state_dict()
}

torch.save(checkpoint, args.model_dir + "/pt_model.pt")
`
Thanks for the help!

Hi! For questions like this, it would be best to ask this on the [forum](https://discuss.huggingface.co/). We use issues for bug reports and feature requests. Thanks!

To answer your question: The [sharing custom model guide](https://huggingface.co/docs/transformers/master/en/custom_models) might help with this .<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>@christianjoun did you solve this problem?