yes, training loop is the loop that iterates over the different batches.
These buffer should not require grad or perform operations linked to things that require grad. If they do and you don’t want these to be part of the backward, use the torch.no_grad() context manager.