I want to modify a value of a model parameter, test the accuracy, then restore the original value:
#save the parameter values
orig_params = []
for n, p in model.named_parameters():
	if n == 'conv1.weight':
		print('\nInitial values:\np: {}'.format(p.data.detach().cpu().numpy()[0, 0, :1]))
		print('conv1.weight.data: {}'.format(model.conv1.weight.data.detach().cpu().numpy()[0, 0, :1]))
	orig_params.append(p.clone())

#modify conv1 weights:
model.conv1.weight.data.add_(torch.cuda.FloatTensor(model.conv1.weight.size()).uniform_(-0.05, 0.05))

#restore the original values:
for orig_p, (n, p) in zip(orig_params, model.named_parameters()):
	if n == 'conv1.weight':
		print('\nAfter modifying conv1.weight.data:\np: {}'.format(p.data.detach().cpu().numpy()[0, 0, :1]))
		print('conv1.weight.data: {}'.format(model.conv1.weight.data.detach().cpu().numpy()[0, 0, :1]))
		print('orig_p: {}'.format(orig_p.data.detach().cpu().numpy()[0, 0, :1]))

	p = orig_p.clone()

	if n == 'conv1.weight':
		print('\nAfter restoring p:\np: {}'.format(p.data.detach().cpu().numpy()[0, 0, :1]))
		print('conv1.weight.data: {}'.format(model.conv1.weight.data.detach().cpu().numpy()[0, 0, :1]))
		print('orig_p: {}'.format(orig_p.data.detach().cpu().numpy()[0, 0, :1]))

Note that to modify weights of conv1 layer I’m using model.conv1.weight.data, but to restore the original value I’m using the corresponding element in model.named_parameters() list.
For some reason, this does not work:
Initial values:
p:                    [[-0.045 -0.048 -0.05  -0.089 -0.085]]
conv1.weight.data:    [[-0.045 -0.048 -0.05  -0.089 -0.085]]

After modifying conv1.weight.data:
p:                    [[-0.019 -0.014 -0.057 -0.056 -0.099]]
conv1.weight.data:    [[-0.019 -0.014 -0.057 -0.056 -0.099]]
orig_p:               [[-0.045 -0.048 -0.05  -0.089 -0.085]]

After restoring p:
p:                    [[-0.045 -0.048 -0.05  -0.089 -0.085]]
conv1.weight.data:    [[-0.019 -0.014 -0.057 -0.056 -0.099]]
orig_p:               [[-0.045 -0.048 -0.05  -0.089 -0.085]]

You can see that even though p and conv1.weight should point to the same memory, and changing conv1.weight.data does change p.data, changing p does not change conv1.weight.data. I’d like to understand what’s going on here.