I am sorry, I don’t know what is the correct name of what I want. I want to compare value of parameter at the starting point of learning and at the later stage of learning. In order to do so, I used detach() to parameter when learning is about to start. But it seems the detach() value is still changing as learning goes on. Then, I cannot compare original value and trained value of parameter since original value is changing.
I wrote a simple code to reproduce this problem. There is a simple model and it has just two parameters, self.p1 and self.p2. Those values are initially saved using detach() as p1_d and p2_d. And after simple training finishes, I compared values of original p1_d&p2_d and trained p1&p2. It seems the detached values are changed.
How do I get unchanged instance of parameter?
import torch
import torch.nn as nn
import torch.optim as optim

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.p1 = nn.Parameter(torch.tensor([1.0]))
        self.p2 = nn.Parameter(torch.tensor([100.0]))

    def forward(self):
        return self.p1 + self.p2

model = Model()
criterion = nn.MSELoss()
optimizer = optim.Adam(list(model.parameters()), lr=0.1)

print("before start")
p1_d = model.p1.detach()
p2_d = model.p2.detach()
print(p1_d)
print(p2_d)

for epoch in range(10):
    loss = criterion(model(), torch.tensor(20.0))
    print('loss: %d' % loss)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()


print('end of training')
print('original_values: ')
print(p1_d) #not 0.0. the value has changed.
print(p2_d) #not 100.0. the value has changed.
print('trained values')
print(model.p1)
print(model.p2)
```