How can we assign an external tensor to the nn.Parameters of a network and keep the gradient flowing out of the network back to the external source of the parameters? The problem is that when assigning the tensor to the weights of a layer, I can’t find a way to make autograd see that the assignment to the weight is a slice from a larger tensor of parameters.
Create parameters

model = nn.Sequential( nn.Linear(1, 1) ).to(device)
theta = torch.tensor([[1.0,2.0]], requires_grad=True).to(device)

Select slice of theta to be assign as weight

w = torch.reshape(theta[0][0:1], model._modules[‘0’]._parameters[‘weight’].shape)

This returns a tensor with the right grad_fn:
tensor([[1.]], device=‘cuda:0’, grad_fn=< AsStridedBackward >)
Assignment with nn.Parameter does not work since I think there is clone() when constructing

model._modules[‘0’]._parameters[‘weight’] = nn.Parameter(w)

This returns a parameter with no grad_fn:
Parameter containing:
tensor([[1.]], device=‘cuda:0’, requires_grad=True)
Assignment with .data does not work either

model._modules[‘0’]._parameters[‘weight’].data = w

This returns a parameter with no grad_fn:
Parameter containing:
tensor([[1.]], device=‘cuda:0’, requires_grad=True)
I need to find a way to keep the Backward assignment when passing the new tensor into the Parameter