Is there any updates on this issue? I am facing the same problem, because I train a model multiple times with different trainers, and every time a new trainer is initialised the gpu memory usage increase.