Hi,
So I’m trying to make a neural network that copies over trainable weights from an ‘ancestor’ network that is almost similar. This is how I currently do it:
 list(self.parameters())[layer_number * 2][:-1, :] = params[layer_number * 2]
Here ‘params’ is a list of the trainable parameters of the ancestor network.
params = list(self.parameters())
My issue is that when I print out the tensors of the NEW network, I see grad_fn=<CopySlices> in the tensors that were copied over.
[Parameter containing: tensor([[-0.0237,  0.1563,  0.0842, -0.3308, -0.3521],         [ 0.1558, -0.0448, -0.4337, -0.3235, -0.3658],         [-0.1654,  0.1849,  0.1011,  0.2428,  0.3796]], grad_fn=<CopySlices>),  Parameter containing: tensor([-0.4441, -0.3973, -0.4269], grad_fn=<CopySlices>),  Parameter containing: ...
The parts of the new network that have completely new values (no copied over values at all) have ‘requires_grad=True’.
Parameter containing: tensor([ 0.1209,  0.0306,  0.5345, -0.3023], requires_grad=True), Parameter containing: tensor([[ 0.0771, -0.1399, -0.1571,  0.2052],         [-0.1313,  0.0706, -0.0783, -0.3379],         [-0.1491,  0.3440,  0.4240,  0.0006],         [-0.1941, -0.0320, -0.3423, -0.4040],         [-0.1626,  0.3006, -0.4289, -0.1277]], requires_grad=True), Parameter containing: tensor([ 0.1358, -0.0119,  0.4847,  0.1065, -0.2739], requires_grad=True) ...
Does this mean that the values that were copied over to the new network are untrainable? What can I do to make the copied over values also show ‘requires_grad=True’ (and not  grad_fn=<CopySlices>)? And I want to get rid of the ancestor weights entirely from the computation graph. All I want is to simply transfer over the ancestor values and nothing else. Please help.