Depending on the original model architecture, you should be able to access the internal modules directly e.g. via:
emb = model.encoder.embedding
fc = model.fc

These attribute names are just randomly picked by me and you would have to look into the model architecture to get the corresponding names.
Once you have all modules, you could pass these to your custom model and just reuse them in the forward method.