@ptrblck Thank you for your reply!
I have checked the docs. But I still have some confusion over the memory. I met an OOM error. RuntimeError: CUDA out of memory. Tried to allocate 1.53 GiB (GPU 0; 11.75 GiB total capacity; 5.97 GiB already allocated; 449.69 MiB free; 8.99 GiB reserved in total by PyTorch).
It seemed that I still have about 3G (8.99GiB-5.97GiB) cached memory to allocate tensors, but the allocation failed. So I guess 1.53 GiB tensors might be moved to free memory and then removed to cache memory, or the cached memory was occupied by Pytorch internal context management but why could it occupy so large space.
Did I make anything wrong? Thanks!