Even if that same process can reuse the GPU memory, it doesn’t look like other processes can. I’m running into a similar utilization concern.
Another process will run into Out of Memory errors, while the original process keeps the GPU memory even after it is done using it.