I suspect that some_var[:,:,i,j] = stuff counts as an inplace operation.
If I have understood correctly what your code is doing then most of them are unnecessary, and the last one sample[:,:,i,j] = images[:,:,i,j] doesn’t need gradients.
Would this work?
for i in range(picSize):
    for j in range(picSize):
        optimizer.zero_grad()
        out = net(sample)
        residual = (images[:,:,i,j] - out[:,:,i,j])**2
        loss = (residual.sum()/(bs * 3))
        pdb.set_trace()
        loss.backward(retain_graph = True)
        optimizer.step()
        sample.data[:,:,i,j] = images[:,:,i,j]