Detaching e.g. the output from the computation graph would set the requires_grad attribute to False, which should not happen “automatically” after a specific number of iterations.
Are you using conditions inside the forward method, which could pick another code path and could either directly detach tensors or use non-differentiable operations, or any data-dependent control flow which could do the same?