This bug really drives me crazy. Somehow my friend advised me to try commenting out ‘torch.autograd.set_detect_anomaly(True)’, and the memory leak issue magically disappears. But neither of us has a clue why it is the case, though…