Are there any recommended methods to clone a model?
I’m interested to clone a model for various reasons (makes it easy to stack untied versions of the same model for instance). Any recommended methods for doing so?
mrdrozdov
apaszke
Thanks! Seems like a reasonable approach.
mrdrozdov
Ended up going with something like: <SCODE>model_clone = model_cls(**kwargs)
model_clone.load_state_dict(copy.deepcopy(original_model.state_dict()))
<ECODE> Works well for me!
mrdrozdov
No need to deepcopy the state dict. Its contents won’t be assigned, but copied into the clone.
apaszke
I have a situation where I am copying weights back and forth between two instances of the same model. Unfortunately, copy.deepcopy is not working for me. I am having to do: <SCODE>mp = list(model.parameters())
mcp = list(model_copy.parameters())
n = len(mp)
for i in range(0, n):
    mp[i].data[:] = mcp[i].data[:]
<ECODE> While this is fine, I wonder why deepcopy function is not working.
rk.epfl
Did you figure it out? I am having the same problem
damaru
I did not find a different way when I had this problem. But more recently, I used the “copy_” operation to copy weights layer by layer from one model to another: <SCODE>mydict = mymodel.state_dict()
layer_names = list(mydict)

# Now, to copy values for a particular layer using the name or index of it:

mydict[layer_names[index_of_layer]].copy_(some_data_with_matching_shape)
<ECODE> If there is a better way, I would be happy to learn.
rk.epfl
What happens if I do this: <SCODE>hNetModel = Model()
    for trainBatch, trainLabels in hTrainLoader:
        <Train the Model by a Function>
        modelEvaluationMetric = hNetModel(Validation)
        if(modelEvaluationMetric < bestModelMetric):
            hBestModel = hNetModel
<ECODE>
Royi
albanD
<SCODE>hNet in NetList
hNet = TrainNet(hNetModel)
modelEvaluationMetric = hNetModel(Validation)
        if(modelEvaluationMetric < bestModelMetric):
            hBestModel = hNet
<ECODE> I thought at least when something gets back from a function it is a different copy of it (Yea, I’m not so experienced with Python).
Royi
just to make your answer clear you mean: <SCODE>new_mdl = copy.deepcopy(model)
<ECODE> right?
Brando_Miranda
why is deep copy not working for you? in what way is it not working compared to what u expected?
Brando_Miranda
Does something inspired from: or not work for you?
Brando_Miranda
Hi, copy.deepcopy(model) works fine for me in previous PyTorch versions, but as I’m migrating to version 0.4.0, it seems to break. It seems to have something to do with torch.device. How should I do cloning properly in version 0.4.0? (torch.device device) (str type, int index)
X-czh
Deepcopy is not working for me. I have a function train(model) which returns the trained model, model_trained = train(model_untrained). However as result both are trained at the end, but I want the model_untrained to be unchanged. So I tried to deep-copy the model_untrained inside the function before the training loop, but It is not working – the model is not trained correctly. Any idea why is it happening?
kuzand
ptrblck
Yes I am training the copied model. You are right about the optimizer, I was passing the original model parameters to it. Thanks for spotting it!
kuzand
shivamsaboo17
<SCODE>import pickle
copyed_model = pickle.loads(pickle.dumps(model))
<ECODE>
52153d3b7bd2d9066500