You are storing a few tensors in your code such as label, out etc.
How large are these tensors and how many are you storing? I’m not sure if how much memory your system has and if the memory increase is expected, since you are indeed storing some metrics.