Hi,
I have trained a model, and then I implement inference with it. After the first inference, the model takes a large amount of memory. Then even though I no longer feed inputs to it, it still takes up these memories. I tried torch.cuda.empty_cache, but it still cannot  shrink the memory usage to the amount before the first inference. How could I unload the model and keep the process going when no input is fed, and reload it once new inputs are available?