Thanks for the explanation, it is very useful!
But I have another question, In my case, L is updated at each timestep, and the output at each timestep is calculated based on the new L and weights W.
loss = 0
L = Variable(torch.Tensor(i_size, j_size))
W = Parameter(torch.Tensor(20, 20)) #fake size
for t in range(time_step):
   for i in range(i_size):
       for j in range(j_size):
           L[i, j] = func(L[i,j]) # a function of wrt old L
   out = get_output(L, W)  #output computed from L and weights W
   loss  += loss_func(out, label[t])


In this case, can I still get gradient of loss wrt the weights W using autograd? It seems that L is overwritten at each timestep.