Solved by ptrblck in post #5 
              
              
                Here is an example: 
var1 = torch.rand(128, 128, requires_grad=True)
m1 = var1.new_empty((4, *var1.shape)) # intermediate variable
a = torch.linspace(1, 10, 4).reshape(4, 1, 1)
data = torch.randn(128, 128)

for i in range(10):
    m1.detach_()
    m1[:] = torch.exp(var1[None, :] ** 2)
    l = (m1 - …