In one of my trials, I try to compute loss for each of my data then collect the “prob” in a list. Then, in each iteration(time step), I try to update my net. But with RunError: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 2]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
But, I don’t know where is the wrong lies, since I don’t have any in-place operation. Could any one help me?
My code Like:
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import matplotlib.pyplot as plt
from torch.distributions import Categorical
from torch.nn.utils import clip_grad_norm_

torch.autograd.set_detect_anomaly(True)

class Policy(nn.Module):
    def __init__(self, n_input, n_output, hidden_dim = 128):
        super(Policy,self).__init__()
        self.n_input = n_input
        self.n_output = n_output
        self.lin1 = nn.Linear(self.n_input, hidden_dim)
        self.lin2 = nn.Linear(hidden_dim, self.n_output)
    
    def forward(self,state):
        x = F.relu(self.lin1(state))
        x = self.lin2(x)
        return x

p = Policy(4,2)
lists = []
optimizer = optim.Adam(p.parameters(), lr=0.01)

#psudeo data
data = [[1,2,3,4],[2,1,3,4],[3,1,3,4]]
data = torch.tensor(data).float()

for i in range(data.shape[0]):
    x = data[i,:].unsqueeze(0)
    prob = p(x).max()
    lists.append(prob)

#BIG PROBLEM HERE!!!
for t in range(len(lists)):
    print("t:",t)
    loss = -lists[t]
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()