Hi,
The difference between putting it inside or outside the loop is the difference between a batch of size 1 or len(recorder.state_tape). So it is possible that in your case, this larger batch size is needed for the training to be stable and the model to learn?