Hi,
The thing is that at each iteration, you increase the input size. So your state (and layers) are bigger and bigger with the depth.
The input of the linear is needed to compute the backward pass. So you won’t be able to reuse a Tensor and override it with new values.