Make sure to avoid storing any tensors which are attached to the computation graph.
I.e. if you are appending any tensor to e.g. a list or are accumulating them via e.g. loss += batch_loss, check if the used tensor has a valid .grad_fn and if so .detach() this tensor before storing it (assuming this fits your use case).
If that’s not the case, then check if your system might be throttling due to e.g. overheating.