The old value in out1 gets deleted since there are no longer any references to it. It holds onto all the internal state needed to compute the gradient, so when it gets deleted that internal state gets deleted too.