Hey Daniel, just seeing this.
If you are allocating memory on the GPU in a custom function, you will want to call empty_cache() right after the function to clean up any memory from the orignal tensor constructed within the function. As long as you clean up after each function call, you do not need to worry about deleting any memory. By calling empty_cache() after the reassignment of A, you clean up any unused memory in cache.  Note that if you do not do this, it is not a memory leak. Emptying the cache only frees the memory pytorch is using. In reality pytorch is freeing the memory without you having to call empty_cache(), it just hold on to it in cache to be able to perform subsequent operations on the GPU easily. You only want to call empty_cache if you want to free the GPU memory for other processes to use (other models, programs, etc)