Add MVP model
### Model description

Hi, I want to add my model MVP to hugging face. My model is very similar with BART. The difference is that it has pre-trained soft prompt in the format of Prefix-tuning.

We have pretrained a series of prompts combining with backbone transformer. And users can choose which prompt to load. So I wonder where I can change the behaviour of loading which prompt?

To put it simply, our model is composed of a big nn.Transformer and a small nn.Linear. We have pre-trained a Transformer and _**n**_ Linear. Users can choose which Linear to load.
One solution can be that, we upload _**n**_ chechpoints, including the common Transformer and respective Linear. And users just write `model.from.pretrained('model1/2/3')` to load which prompt. However, the it is a bit waste to upload and download the common Transformer **_n_** times.

So I wonder how I can solve it with a simple method. Such as our uploaded model is consist of Transformer and **_n_** prompts, and users can choose which prompt to use through config.





### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_
Hello! I think this is a situation where the code-on-the-hub approach is likely to be the best way to share it! See this guide: [Sharing custom models](https://huggingface.co/docs/transformers/custom_models).

This will give you the most amount of freedom with regards to your model.

cc @sgugger <|||||>Thank you! I will take a look.<|||||>Great! Please let us know if you run into any issues, or if anything is unclear from the guide. Thanks!<|||||>Hi, sorry I still don't know how to handle a situation like mine.

My model is 
```
(
BertModel()
nn.Linear()
)
```

And my checkpoint is composed of
```
(
BertModel()
'a': nn.Linear()
'b': nn.Linear()
'c': nn.Linear()
)
```

And my expected effect is that user can determine which Linear to use through the config, such as `XXX.from_pretrained('my_model', linear='a' or 'b' or 'c')` to load according linear.
However, the model initilization is before the model loading. So I don't know how to solve it.<|||||>A quick solution would be to have three architectures, one with `a`, one with `b`, and one with `c` linear layers and load the one you want accordingly.<|||||>Do you mean I upload three models, named my-model-a, my-model-b and my-model-c, then load the one I want?

<|||||>You can upload one single checkpoint, but three different architectures that each use the `a`, `b`, and `c`. Similarly to models trained on sequence classification (or any other task): they can be loaded in base models, in question answering models, etc. It will only load the layers it needs, and ignore the rest.<|||||>Thanks for your response! I understand what you mean.

Actually, my situation is a little different from the task head. The prompt is the part of the encoder and decoder, and I have 8 different prompts rather than 3. So I wonder if there is any other way, considering they have the same architecture.