Thank you for your response!
Yes, since I would like the allocator to directly release the memory(the inefficient sollution), or efficiently reuse the allocated memory regions (I have not observed this behavior in Impl #3), so is there any other allocator implementations, and their documentation? I am not very familiar with this part in pytorch.
BTW, the intermediary tensors are needed because I need to prevent users from unintentionally modify the stored tensor data.