saving pretrained models that were obtained from another model
I am trying to save a pretrained model that I created from RobertaForQuestionAnswering by changing some layers.
However, when I load the model with from_pretrained, my new layers disappear.. It makes sense since binary seems to save only the model weights, but I wonder if there's a way to work around this?
@dar-tau even though the file stores only weights, but if you added more layers those weights should be saved right? And when you try to load that model into RobertaForQuestionAnswering without your extra layers it should fail. <|||||>Thanks for your reply. 
I'm using AutoModelForQuestionAnswering.from_pretrained(...) and not RobertaForQuestionAnswering (and as a matter of fact I'm actually replacing layers rather than just adding some).

What I am aiming for is a way to make it "forget" it originated from Roberta, and save the entire model.
My desire is that it will be loadable with from_pretrained(..) and semantically equivalent to:
torch.save(model, "file.pt") 
model=torch.load("file.pt")<|||||>@dar-tau I don't think this is something you can do with transformers, you need to probably do it using torch directly. You might want to check the config.json file which is saved.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.