Inplace matrix modification
ypxie
Yes it is. This should work: <SCODE>L = Variable(torch.Tensor(i_size, j_size))
# it's important to not specify requires_grad=True
# it makes sense - you don't need grad w.r.t. original L content,
# because it will be overwritten.
for i in range(i_size):
    for j in range(j_size):
        L[i, j] = # compute the value here
<ECODE> But beware, it might be very very slow! Not only because you’ll be looping over elements in Python, but also because it will involve a lot of autograd ops to compute this, and there’s a constant overhead associated with each one. It’s not a huge problem if you’re doing relatively expensive computation like matrix multiplication or convolution, but for simple ops it can be more expensive than the computation alone.
apaszke
<SCODE>loss = 0
L = Variable(torch.Tensor(i_size, j_size))
W = Parameter(torch.Tensor(20, 20)) #fake size
for t in range(time_step):
   for i in range(i_size):
       for j in range(j_size):
           L[i, j] = func(L[i,j]) # a function of wrt old L
   out = get_output(L, W)  #output computed from L and weights W
   loss  += loss_func(out, label[t])

<ECODE> In this case, can I still get gradient of loss wrt the weights W using autograd? It seems that L is overwritten at each timestep.
ypxie
apaszke
Tahnks！ That’s exactly what I want to know.
ypxie