Hi,
Thanks a lot for your reply.
Concerning the dicts, means is initialized as a defaultdict(float) and monitored_quantities is defined as {'loss': loss.item()}. So both dicts are dicts of Python floats, and none of them contain any tensor, whether on CPU or GPU.
Moreover, all operations I realize inside of the for loop are written in the snippet and in particular nothing is printed in the loop. I did not include the code of the step function because I assume that local tensors created in this function are out-of-scope and thus should be removed from GPU memory when I call torch.cuda.empty_cache(). Am I right on this point?
EDIT: I tried replacing loss.item() by loss.detach().cpu().item() in the monitored_quantities computation but it changed nothing.